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-'ABSTRACT 

/ This report presents the final scope, methods,, and 

-*esul|s of the evaluation of proficiency measures in occupational 
therapy. The intended purpose of the investigation was to evaluate 
and analyze the reliability and validity of measurements that are 
predictive of competence and proficiency at entry levels in 
occupational therapyi Each level of the Proficiency Examination (PE) 
was evaluated with a number of independent measures of occupational 
therapy knowledge .and practice. Existing measures^ the National 
Certification Examination (NCE) , and the Field Work Performance 
Report (FWPR) , were compared with the therapist level examination, 
*!l!L Qu itative Assessment of Field Work Competency' (QAFWC) . The 

was 

. pel 

_____Qjia-li U l± Assessment of ^ield Work Performance (QAFWP) , a measure 
of field work performance at the assistant lr jr e l, was developed as , * 
independent measure for the evaluation of th assistant level PE. 
- Information on the data collection phases; general descriptive data; 
the factor analysis of the PE and the NCE; the; relationship between' 
grade point average and the PE, QAFWC , and FWPR; anc the comparative 
score performance between therapists and assistant therapists on the 
therapist level PE is given. Conclusions and recommendations are made 
based on the data. (RC) \ 
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a measure of frequency of performance of 1 reguired skills, 
3d to include a gualifc&tive dimension of performance. The 



) 



Documents acquired by ERIC include many informal unpublished materials not available from other sour<4> ERIC makes every 
effort to obtain the best copy available. Nevertheless, items of marginal reproducibility are often encounterfeoand this affects the 
quaUty of the microfiche and hardcopy reproductions ERIC makes available via the ERIC Document Reproduction Sasvice (EDRS). 
EDRS a i>,>t responsible for the quality of the original document. Reproductions supplied by EDRS are the best that can be made from 
lainal. • 



ERIC 



\ 

\ 

\ 



SCOPE OF INTEREST NOTICE 
The ErtlC f*ciUi v h«i aUltfn ^ 
Ihii document «c [IMOeuj >g 



1W. 



. of >nt»r«t ro ih* 0***10$. 

tout*) notiK) to the Ifxfc* 
««fl *houW vitrei ihe.i io#o«» 
prmit ot 



EVALUATION 
/OF. 
U1ES OF PROM 



1EM 

M OCCUPATIONAL THERAPY 



HMAL REPORT 

Contract #231-75-0205 



US DEPARTMENT OF HEALTH. 
EDUCATION A WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION 

I 

THIS DOCUMENT MAS BEEN "RjE PRO- 
DUCE 0 EXACTlY AS R v ECE:VED/FROM 
THE PERSON OR ORGANISATION ORIGIN- 
ATINGIT POINTS OF VIEW CW OPINIONS 
STATED DO NOT NECESSARILY REPPE- 
SE NT OF P iCl A t NATIONAL INSTITUTE O p 
EDUCATION POSITION OR POl'iCY 



If} 



By 

The American Occupational Therapy Association 
6000 Executive Boulevard 
Rockville, Maryland 20852 



s 



2 



o 

ERIC 



/ 



/ 

FINAL REPORT 
IK 

OF 



/ 

/ 



\ 

\ 



\ 

\- 
\ 



\ 



EVALUATION OF MEASURES OF PROFICIENCY 
IN OCCUPATIONAL THERAPY 

Contract #231-75-0205 



Submitted to:- 

Department of Health, Education, and Welfare 
Health Resources Administration 
Bureau of Health Resources Development 
Division of Associated Health Professions 



The American Occupational Therapy Association 
6000 Executive Boulevard 
Rockville, Maryland 20852 



August 31 , 1976 

\ 



9 

ERIC 



PROJECT STAFF 



The Ame rica n Occupational Ther apy Asso ciat ion 

Diane Shapiro, MA, OTR - P»ojkct Director 
Sandra Prevel - Administrative Assistant 
Mary Gecrsatsos - Secretary ( 



Professional Examination Service 

Lillian Terris, Ph.D/ * Project Director 

Jane Mathews, MS, *PT - Assistant Project Directo 

Philip Ferrara - Program Associate 

Barbara Bernstein - Administrative Assistant 



Project Officer 

i 

Margaret 'toil son, Ph.D. 

Manpower Utilization Branch 

Division of Associated Health Professions 

Department of Health, Education, and W^Ware 



ACKNOWLEDGEMENTS 



While in the midst of completing requirements for graduation, 
several hundred recent members of the profession volunteered to 
write the Proficiency Ixamination. They are gratefully acknowledged. 
The proctors, curriculum and program directors, and field work 
supervisors are also acknowledged for the many hours of hard work 
contributed to the project. 

i 

The project director sincerely thanks the AOTA and- PES staffs for 
their- individual and coordinated efforts. A special note of appreci- 
ation is extended to Sandra Prevel and Barbara Bernftein. 

The participation of Carol Nathan, Chair of the Advisory Committee, 
and ail of the members ''are to be commended for their long standing 
interest and devotion. The staff warmly appreciates, their involve- 
. ment. ^ 

/..■• ■ ■ ■ - * ' • . ■ ... ' 

Simme Cynkin, consultant to the project and creator of the qualitative 
field work instruments, is acknowledged not only for the fine job 
she has done, but also for the speed in which it occurred. Jane 
Mathews and Lillian Terris are thanked both for inspiring Simme and 
also for numerous other contributions. 

An honorary degree in occupational therapy should be presented to 
Philip Ferrara; he can now, after months of responsibility for the 

r» *■ 



I 



statistical analyses, define occupational therapist more clearly 
than most occupational therapists. 

A message of gratitude to Mary Georgatsos, Dale Brown, and Barbara 
Kattaron, for their assistance in preparation of this report, and 
for their many other contributions! 

A special thanks to Sally Ann Henry, The Psychological Corporation, 
for assisting in the administration of the examinations. ' 

And finally, thanks to Margaret Wilson, Project Officer, for her 
patience, understanding, flexibility and advice. 




6 



in 



ERIC 



TABLE OF CONTENTS 

Page 

Staff i 

Acknowl edgements 1 j 

Index of Illustrations v 

Introduction - Background and Purpose 1 

Scope of Work 4 

Methods 7 

Problems jEncountered 18 

Results 21 

Summary and Recommendations ./ 62 

Future Use of Proficiency Examinations and Pro- 68 
posed Use of Performance Instruments 

APPENDICES 

Advisory Committee Members and Consul tants Appendix A 

OT and OTA Curriculum Directors Appendix B 

Pro6tors N Appendix C 

Blueprints Appendix D 

Field Work Performance Report and Rater's Guide Appendix E 

Qualitative Assessment of Field Work Competency 

and Guidelines for Raters Appendix F 

Qualitative Assessment of Field Work Perform- 
ance and Guidelines for Raters Appendix G 

Evaluation Results and Fbrms^From QAFWC and 

Q AFWP ' Appendix H 



♦Forms have been removed from the 
report by the forwarding agency: AOTA 



IHhi'K OF ILLUSTRATIONS 

Chart Summary of fata Collected, Therapist Level 

2 « Swrmary of Data Collected, Assistant Level 

Table Descriptive Data Summary: Proficiency Examination 

Performance of Occupational Therapists. 

/ 

2 « , Descriptive Data Summary: National Certification 

Examination, Qualitative Assessment ,of Field Work 
Competency, and FWPR. 

3 « Intercorrelatlons Among Subareas of the Prqfldency 

Examination for Entry Level Occupational Therapists. 

4 - Intercorrelatlons Among Subareas of the National 
Certification E/ami'i*t1on. 

, j 

5 - Intercorrelatlons Between the Subareas of the 
National Certification Examination and the Subareas 
of the Proficiency Examination* 

6 - Intercorrelatlons Between the Qualitative 
Assessment of Field; Work Competency and the Field 
Work Performance fteport (FWPR). 

7 - Intercorrelatlons Between the Qualitative Assessment 
of Field Work Competency and the Proficiency 
Examination for Enjry Level Occupational Therapists. 

8 - Intercorrelatlons iBetween the Field Work Performance 
/ Report (FWPR) and/ the Proficiency Examination for 

Entry Level Occupational Therapists. 

9 - Intercorrelatlons Between the Qualitative Assessment 
of Field Work Competency and the Field Work 
Performance Rqport (FWPR), as a function of the 

three process categories of the Proficiency Examination 



10. 



Intercorrelatiops Between Qualitative Assessment of 
Field Work Competency and the National Certification 
Examination. 



11 • Intercorrelatitfhs Between Field Work Performance 

Report and the National Certification Examination. 

12. Rotated Factors (Varimax) for the Proficiency 

Examination Subaress. 



13, 



Rotated Factors (Varimax) for the National 
Certification Examination Subareas 



Table J4. 
15. 
16. 

; i7. 

18. 

19. 
20. 

21. 

Figure 1. 

2. 
3. 
4. 
5. 

6. 

7. 



Rotated Factors (Varimajc) for the National Certification 
and Me Proficiency Examination Subareas Combined. 

Clinical Score Performance By Field Work Setting 
at the Occupational Therapist Level. 

Correlation Between Grade Point Average and Scores 
on proficiency Examination, QualiUtive Assessment of 
Field Worfc, and Field Work Performance Report. 

Descriptive Oata Summary: Occupational Therapy . 
Assistant Proficiency Examination 



Intercorrelations Between the Qualitative 
Assessment for OTAs and the Proficiency 
Examination for Entry Level Occupational 
Therapist Assistants 

Rotated Factors (Varimax) for the OTA Proficiency 
Examination Subareas , 

Descriptive Summary Data: Comparative Performance of 
Occupational Therapists and kcupational Therapist 
Assistant Sample Populations on Form 1 of the Proficiency 
Examination fo»* Entry-Level Occupational Therapists. 

Discriminant Analysis Between Entry-Level Occupational 
Therapists am? Occupational Therapist Assistants on the 
Occupational Therapy Proficiency Examination, 

Frequency Distributions of Total Scores for Occupational 
Therapists and Occupational Therapy Assistants on Occupational 
Therapy Examination. - 

Frequency Distributions of Scores for Occupational Therapists 
. and Occupational Therapy Assistants by Subarea: Self Care. 

Frequency Distributions of Scores for Occupational Therapists 
and Occupational Therapy Assistants by Subarea: Work. 

Frequency Distributions of Scores for Occupational Therapists 
and Occupational Therapy Assistants by Subarea: Play/Leisure. 

Frequency Distributions of Scores for Occupational Therapists 
and Occupational Therapy Assistants by Subarea: Motor 
Functioning. ; 

Frequency Distributions of Scores for Occupational Therapists 
and Occupational Therapy Assistants by Subarea: Sensory- 
Integrative Functioningu 

Frequency Distributions of Scores for Occupational Therapists 
and Occupational Therapy Assistants by Subarea: Cognitive 
Functioning. 



Page 40 
42 
43 

— '1 4S 
46 

4/ 
49 

50 

r 

51 

52 
53 
54 
55 

56 

57 



vl 



9 



Figure 6. 



9. 



Frequency Distributions of Scores for Occupational Therapists ?aqe SB 
and Occupational Therapy Assistants by Sj/barea: Psychological 
Functioning. - 

Frequency Distributions of Scores for Occupational Therapists & 

and Occupational Therapy Assistant* by Subarea; S$c*at 

Functioning. 



' 10. 



Frequency/Distributions of Scores for Occupational Therapists 
and Occupations: therapy Assistants by Subarea: iff e Space. 



v1i 



10 



ERIC 



BACKGROUND 



"'' '• s- 



The American Occupational Therapy, Association was Swarded a two -y$*r 
con tract {NO^ -AH-241 72) from the Department of Health, Education, and 
Welfare, Public Health Service, National Institute of .Hej^h, Bureau 
,of Health Manpower Education, in 1971, to delineate We roles and 
functions of occupational therapy personnel, the*«Profes- 
sional Examination Service was then awarded^n 18-month contract 
.(N01-AH-34G63) from the Department j^Hea 1th, Education^ and Welfare", 
Hunan Resources Admi ni s t rati on ,\ Bureau of Health Resources Develop- : 

i • : - \ " . • "'■ , : , ' » 

went. Division of Associated Health Professions,' to develop proft- 
ciency examinations for entry-lev^l occupational therapists and entry- 
level occupational therapy ^sistaVts.. These examinations- were based 
upon task inventories derived from the Hies and functions. 

A third contract was then awarded to The American Occupational Therapy 
Association for 13 months, beginning (n 1975, from the Department of, 

Health, Education, and Welfare, Human Resources Administration, Bureau 

. ■ ■ \\ . • 

of Health Resources Development, Division of Associated Health Profes- 
sionals, to conduct an evaluation of theWoficiency^aminations. This 
report presents the final scope, methods J and results, of the evaluation 
of proficiency measures in occupational therapy. 



/ 

/■ 
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The support for t^e above three contracts had its base w Public, taw- 

The Division of Associated Health Professions", Heatfh .,. 
Resources Administration, is empowered under PL 91 -SIS j Title H, - 
Secsien 792 (c) (2) tc "efcter-'.into contracts, ..for special projects 
relating ..to training or retraining of allied hea 1 to.;, pensfcnne 1 , incltid-. 
i ng . . . ( if } developing, demonstrating, .or evaluating technfques for 
appropriate recognition { .including equivalency and proficiency test- 
ing «rechanis«ns| of previously acquired training or experience/' 
,f l 31-603, Section 1^23, authorizes the Secretary tc carry fcut a pro- 
gram designed to determine the proficiency of individuals (who do not 
otherwise meet the formal., educations*-, professional, membership, or 
other specif ic -criteria established for determining the qualifica- 
tions of particular health groups) to perform the duties and func- 
tions of those health; groups, ft any individual has ?>een .determined 
qualified, no person er provider using the services of such individual 
to perform such duties and functions' shall be denied payment under 
Title XVIII on the grounds that such an individual is nc4.jual if i^d, 

PU9P0SE 

The intent of this third phase was to evaluate each level of the Pro- 
ficiency Examination (PE) with a" number of independent measures of 
occupational therapy knowledge and practice. Existing measures, the 
National Certification Examination (HCE), and the field Work Performance 

'12 
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Report (FWPR), are presently available for compan • with the thera- 
pist level er.ami natron. 

The, we of the NCE and FWPR as independent measures xas not seen as 
sufficient for the purposes of this study, becaus- 'hey do not ade- 
quately reflect the roles and functions as defined by the results of 
the first ffhase. The FWPR is a • <*easure of .the frequency of" performance 
-of required skills. The • assur... ;:ion that competency in occupational 
therapy couVd be assessed via a measurement of frequency was questioned. 
The members of a pre-contract conference proposed that the FWPR be 
revised to- "include a qualitative 4imens*ion of performance and to 
reflect the roles , and functions' delineated under contract; NO* -AH- 241 72. 

AOTA did not have any. measure. of field work performance at -the assist- 
ant level. Therefore, the -rtveVopnfent at? independent measure for 
the evaluation of. the -as*, ^start level. Pt was included in the scope of' 
this contract. This instrument, the Qualitative Assessment of Pield 
Work Performance (QARrfP) was based upon the roles and functions deline- 
ated under Contract f N01-AH-?4 172. '"' ' 



r 
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The following was to' ! be completed by the contractor, The American 
Occupational -Therapy Association (AOTA) and the subcontractor, Pro- 
fessional Examination Service (PES) . '.• 

ml 



ADVISORY COMMITTEE 



Most of the members of the Advisory Committee for the previous two 
proficiency testing contracts; N01 -AH-34063 and NOI-AH-24173 were 
asked to join the new committee. The members represented institu- ^ 
tions and organizations employing occupational tht-rapy personnel, 
and faculty of technical and academic occupational therapy programs. 
" measurement specialist was also askedNto participate as a consult- 
ant to the committee. The pames and titles of all committee members 
are r . l isted in^Appendix A. The committee met three times during the 

Tonth contract period. The purpose of the committee was to review 
and comment upon the methodology propc^e4 by the project staff and to 
review and help interpret the -data ejected. _ The committee was also 
ked to prepare recommendations to AOTA for use of the examinations 
and/op continued. study. 

PERFORMANCE INSTRUMENTS ■ ' < 

• A qualitative addendum to the FieJiWJoTk^rformance Report (FWPR) 
was to be developed and used as an independent measure of field work 
performance. ^ 

' v ; 

1 i 



For the evaluation of the assistant level Proficiency Examination 
(PE), a qualitative assessment of field work performance was to be 
developed. 

ADMINISTRATION OF PROFICIENCY EXAMINATION 
THERAPIST LEVEL: , 

• A 250-item therap-jst level examination plus selected items from 
the therapist level item pool was to be administered to occupational 
therapy candidates in June 1975. 

• A 150-item form of the therapist level PE was to be administered 
<tp occupational therapy candidates in January 1976. 

ASSISTANT LEVEL: 

• A 250-item therapist level PE and a 250-item assistant level PE 
were to be administered to occupational therapy assistant candidates 
during August and October 1975. 

• A 250-item form of the assistant level PE was to be administered to 
occupational therapy assistant candidates during February and May 1976. 

EXPERIMENTAL DESIGN 

Collect, compare, and analyze the following data: 

# 

• For Occupational Therapy Candidates (OTs): 

15 



1. One to three Qualitative Assessments of Field Work Competency 
(OAFWC) ratings on each student. (Instead of revising the FWPR, 
contract staff decided to develop an additional instrument titled 
QAFWC which would be' used as an addendum to the FWPR. A detailed 
explanation of this begins on page 10.) 

2. Scores obtained from the NCE. 7 / 

3. Scores obtained from the theraoist level PE. 

4. Scores obtained from the employer's rating form. 

For Occupational Therapy Assistant Cahoidates (OTAs): 

1. Scores obtained from an assistant level PE A , 

2. One to three Qualitative Assess-nert Field Work Performance ,QAFWP) 
ratings on each candidate. 

• For OT and OTA Candidates: 

Scores obtained on same 250-item form of therapist level FF . 

Not included in the original Scope of Work, but added by amendment (Oct. 23 
1975) for further evaluative study of the therapist level PE: 

• Comparison of scores obtained by therapist level candidates of the 
QAFWC and FWPR across field work settings. 



• Comparison between the setting of passing ; criteria on the PE, FWPR, and, 
QAFWC. (This comparison was attempted but because no significant corre^la-* 
tions were, found,: this was not pursued.) 

• Comparison of Grade Point Averages (GPAs) with scores obtained from the 
PE, QAFWC, and FWPR. 

16 
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METHODS 

INSTRUMENTS 
THERAPIST LEVEL 

PROF IC IE jOV EX AMINA TION ' 

Both levels of the Proficiency Examination (PE) were developed by the 
Professional Examination Service (PES) under Contract N01-AH-34063. 
Details of the design and pilot testing of the examinations are in 
the fina.1 report of that contract title.') Oc cupational Therapy Profi- 
ciency Examination (N01-AH-34063, January 31, 1 975 ) . 

Each 250-item examination has nine content subareas (illustrated in 
the vertical column of the blueprint, see Appendix D). Those subareas 
are: Self Care (13% of the total), Work (13J&), Play/Leisure (8%), Motor 
(20%) , Sensory-Integrative (13%), Cognitive (8%), Psychological (13%), 
Social (8%), and Life Space (4%). The examination is also divided along 
a second axis (il lustrated in the horizontal column of the blueprint) 
into the following process subareas: Evaluation (30%), Planning (20%), 
and Implementation (50%). Statistical analyses were .performed along 
both of the dimensions of the examination. 

NATIONAL CERTIFICATION EXAMINATION 

The National Certification Examination (NCE) is the examination used 
by The American Occupational Therapy Association (AOTA) and, adminis- 
tered by The Psychological Corporation as part of the certification 



requirements for occupational therapists, registered. The other 
"requirements for certification are; graduation from an accredited bac- 
calaureate or masters degree level curriculum, and the successful com- 
pletion of at least six months of field work experience, three months 
of which must be in the area of mental health and three months in a 
physical disabilities area. Many curricula require that graduates 
complete an additional two or three month internship. ,To be eligible 
to write the certification examination the candidate jtiust receive the 
endorsement of the curriculum director. / 
\ 

The 250-item NCE is administered' to approximately 800 candidates twice 
each year. The content distribution of the NCE has been chanqed since 
January 31, 1975. At that time, there were three major categories; 
Basic Sciences with* two subareas—Biolocjhical and Behavioral Sciences, V 
Clinical Sciences with four subareas— Medical -Surgical , Neurological, / 
Orthopedics, and Psychiatry, and Occupational Therapy Application with 
four subareas--Medical -Surgical , Neurological , Orthopedics and Psychiatry 

FIELD WORK PERFORMANCE REPORT 

The { Field Work Performance Report (FWPR) is the official instrument 
used by the AOTA for evaluating field work performance for occupational 
therapy students at the end of each of two or three required intern-- 
ships. There ire 53 items divided into five major performance areas , 
Data Gathering, Treatment Planning, Treatment Implementation , Conmunica- 
x ^tion Skills, and Performance Characteristics. Each item is rated on a 
four-point scale, ranging fron one to four points. 



\ ^ 
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Criteria for selection of points is based upon frequency of perfor- 
mance. Performance of a competency 0-25% of the time expected/ 
required equals one point, 26-50% Equals two points, 51-75% equals 
three points and 76-100% equals four points..; (A sample copy of the 
FWPR is in Appendix E.) • /' 

• \ 

As mentioned /in the first section of this report, the project staff 
and consultants, were not totally confident in this instrument for the 
purpose of this evaluation study of the proficiency examination. The 
project staff met with a group of occupational therapy educators and 
clinicians to review the concerns of those people who use the instru- 
ment The therapists and the educators identified two issues of great 
concern to them. (1) Measurement of frequency of performance of a 
desired behavior was not sufficient. For example, some students per- 
formed a snecific behavior 80% of the time, but often would not do it 

i 

.at.-, / and still would receive the maximum score. (2) The four- 

\ 

poini -.ting scale was also found to be a limiting factor since most 

..1 -v. 

students received a score of between three and four as a mean score 
per item. This led to a restricted range of total scores. The 

users of the instrument also reported some positive aspects. The 

\ 

items were considered to represent the varied scope of performance 
areas. The form was considered easy to use. 



The project staff decided not to change or revise the FWPR so as not 
to contaminate the data available on it, but to instead develop a 
separate qualitative instrument that could be used in conjunction 
with the FWPR for the purposes of the evaluation study. 
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QUALITATIVE ASSESSMENT OF FIELD WORK COMPETENCY 

The Qualitative Assessment of Field Work Competency (QAFWC) was devel- 
oped by the project's consultant with the assistance of the PES staff.* 
It was intended to be used as an addendum to the FWPR for the purpose 
of evaluating the therapist level PE. There are a total of eleven items 
in three major performance areas corresponding to the three process 
areas of the Proficiency Examination; Treatment Evaluation, Treatment 
Planning, and Treatment Implementation. A fourth area, Personal Char- 
acteristics, contains three additional items. For each of the eleven 
items, there are five behavioral indices, ranging from one (very weak) 
to five (very strong). The evaluator is asked to select the behavioral 
indicator that most completely describes the student's performance 
for that item. The definition of terms used in the QAFWC is consis- 
tent with the FWPR. All of the supervisors were instructed nojt to allow 
the student to see the form. (The Q^FWC is in Appendix F.) 

EMPLOYER'S RATING FORM 

The QAFWC was used as the employer's rating form. The cover page was 
changed and "employee" was substituted for "student". 

ASSISTANT LEVEL- 

\ 

* 

\ PROFICIENCY EXAMINATION 

\ ' ■ y 

\ ' 

There were three forms of a 250-item assistant level examination. 
Each examination has eight content subareas (illustrated in the ver- 
tical column of the blueprint, see Appendix D ). Those subareas are: 

20 
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Self Care (3055 of the total), Work (30%), Play/Leis-jre (20%), Motor (5%), 
Cognitive (4%), Psychological (4%), Social (5«) 3 and Life Space (2%). 
The examination is also divided along a second axis (illustrated in the 
horizontal column of the blueprini.) into the following three process 
areas: Evaluation (20%),- Planning (10%), and Implementation (70%). 

QUALITATIVE ASSESSMENT OF FIELD WORK PERFORMANCE 

The Qualitative. Assessment of Field Work Performance (QAFWP) was 
developed by the project's consultant with the assistance of the sub- 
contractor's staff as a measurement of performance at the assistant 
level to be used to evaluate the assistant level PE. The AQTA does 
not have an official performance instrument for this level. Each 
approved OTA program use.s their oWn method of performance evaluation. 

In preparing for the development of the QAFWP , the staff collected 
all of the evaluation instruments used by each of the schools. The 
consultant tried to abstract the best psychometric aspects 'of those 
instruments into one format. The format used for the QAFUC was 
favorably assessed by the supervisor/eval uators , so it was also used for 
the QAFWP. Of the four major categories assessed,' three contain a 
total of seven items and correspond to the three process categories 
of the assistant level PE; Evaluation, Planning, and Implementation. 
Ea v ch of the seven items are rated on a scale of five points, from one 
(very weak) t.o five (very strong). There is a descriptive behavioral 
indicator. for each scale. The fourth category, Role Expectations, ; 

21 
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contains four parts with a total of 26 items. Each item is rated on 
the same five-point scale, but there are no behavioral indices. , 

The first draft of the QAFWP was sent to all program directors for com- 
ment. Nine of the 43 responded and those comments were used to revise the 
second^ draft. The second draft was then presented to a group of program 
directors, faculty members, and clinical supervisors at a meeting of 
the 1975 AOTA Annual Conference. The participants carefully reviewed 
the QAFWP and made several suggestions for revision. The final QAFWP 
(see Appendix G ) reflects those revisions. 491 copies of the form were 
then sent to supervisors along with an evaluation form accompanying 
the QAFWP. The responses were tabulated (see Appendix H) and will 
be used for further revision of the QAFWP if the AOTA decides to use 
it or study it for potential use as an official instrument^ - Dis-'' 
cussion of the evaluation and future use of the form appears in the 
section beginning on page 68. 

DATA COLLECTION PROCEDURES 
THERAPIST LEVEL 

Most of the occupational therapy curricula prepare candidates to com- 
plete academic and field work requirements in time for either the June 

t , 

or the January administrations of the NCE. The project staff therefore 
arranged the testing schedule to coincide with these dates. 

22 
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FIRST DATA COLLECTION PHASE 

~) • ■_ ■ 

i •••„ ■ 

On June 21, 1975, one w£eic prior to the- administration of the NCE, a 
two-part therapist level PE (one 250-item examination and an addition- 
al 150 pool items) was administered. All 807 candidates eligible for 
the NCE wr asked to voluntarily participate in the project. 275 
responded to the request with 179 candidates examined at 45 testing 
sites throughout the country. 91 completed at least 95% of both parts, 
and 123 completed at least 95% of the 250-item examination. 

The 275 candidates who expressed willingness to participate were also- 
asked to have the Allowing data forwarded to the PES; 

1) Their scores on the June 28, 1975 administration.^ the NCE. 

2) Copies of the FWPR completed by their field work supervisors. 
(These forms were completed within a six to nine-month period 
prior to the administration of the examinations.) With the candi- 
dates permission, curriculum directors were able to send photo- 
copies of these from the students' files. 63 FWPRs were received 
from the grqup of 123 subjects who also wrote the PE and NCE. 

S) Names and addresses of the current field work supervisors, if 
the candidate was then still completing field work requirements. 
These supervisors were asked to evaluate the student using the 
QAFWC and FWPR. , In addition, supervisors were asked through 
a structured format to give their impressions of the QAFWC. 
Sixty QAFWCs were received from the group of 123 subjects who 
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wrote the PE, NCE, and were also among the 63 who forwarded com- 
pleted FWPRs. 

4) Names and addresses of employers and permission to contact the - 
emplo/er after a six-month pen> A follow-up request was sent 
to candidates who did not know the employer at the time of the 
first. solicitation. The employers were asked to evaluate the sub- 
ject based upon the einployer's rating form. Out of a total of 57 
candidates, responding to both requests, 30 candidates 1 employers 
completed the employer's rating form. 

SECOND DATA COLLECTION PHASE 

Or. January 31 , 1976, a 150-item therapist PE was administered at 23 
sites throughout the country. The examination was reduced from 250 to 
150 items so that a factor analysis of the items could be performed. 

A total of 707 candidates was asked to participate. 220 candidates 
responded positively to the request with 117 candidates completing the 
examination. The 220 willing subjects were asked to grant permission 
to have the following data forwarded to PES: 

1) Copies of the FWPR's from two or three field work assignments, " 
406 were received on 170 subjects. Each subject's scores were 
averaged yielding one score per subject. Based upon the sample 
of 170, 82 individuals also wrote the PE. 
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2) Copies of the QAFWC from their two or three fiejd work t-^&ri- 
ences. The subjects were asked to submit to the- staff the nac.es 
and addresses of their field work supervisors. Copies ot the QAFWC 
with instructions were sent to the supervisors. 46 1' copies of the 
QAFWC were sent to supervisors for the 220 candidates agreeing to 
participate in the study. 256 QAFWCs were returned for a total of 
140 subjects. When more than one QAFWC per subject was received, 
the scores for each subject were averaged. 78 of the subjects who 
sent in at least cite also wrote the PE and were among the 82 who 
forwarded FWPRs 

3) Grade point averages (GP-As) from the curriculum directors. 45 
GPAs were received from those subjects supplying the above data. ' 

ASSISTANT LEVEL 

Students graduate from 41 approved OTA programs at various times 
throughout the year. To secure a maximum number of subjects for both 
data collection phases, , the staff decided to allow each program director 
to select a testing date convenient to the program. T., quality as a 
subject, a student must have completed the academic program and all 
required field work prior to the date of the examination. 

FIRST DATA COLLECTION PH ASE 

Two examinations were administered to a total of 234 subjects during 
August- 1975 and October 1975. ' These examinations were a 250-item form 
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of the assistant level f>£ and the same 250- item form of the therapist 
level PE administered • : cupatiohal therapist candidates on June 21, 
1975. One-half of the ..ejects wrote the assistant level examination 
first and the therapist level examination second. The remaining sample, 
were administered the examination in reverse order. Subjects were 
informed of the levels of each examination at the end of the day 4 ' 191'' 
•completed. at «east 95? of the therapist level examination. Conditional 
data wer»:- collected, J: 

- SECOND DATA COlUCHON PHASE • * 



The sasf* ?S0-Uem forri of the assistant level examination was adminis- 
tered to 183 subjects at 22 different sites between - February and Hay 
1976. 178 subjects completed at ieast' ;; *95*,of ..the examination, the 
subjects {and/or program 'di»-ectorr.) were asked to send to.thfe staff 
the names, and addr-;- -.>>s of thetr field work supervisors, 492 QAfWPs' • 
with- instructions were mailed 'to supervisors, (Note; Each student 
is required to. have two or more field work experiences, accounting for 
the large number of scores. The QAFWFs across field work experiences 
were averaged for each individual candidate.) A total of 140 candi- 
dates' scores were used in the final data analysis. 
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PROBLEMS ENCOUNTERED 

The administrative problems encountered were mainly caused by a very 
limited time schedule and difficulty in obtaining the sample size 
proposed in the initial contract. 

ADMINISTRATION OF THE EXAMINATIONS 

The staff had seven weeksfrfijiL the award of the contract until the date 
of the administratis*^ secure sites, enlist volunteer proctors, locate 
200 subjects, select and print the examination booklets and prepare and 
print the proctor's manuals for the therapist examination. 

The date of administration of the PE presented some problems. The NCE 
was scheduled to be administered on June 28, 1975. Consideration was 
given to selecting a date that would not interfere with the candidates' 
preparatjsin^fiyr^^HCE or would not be too long afterward. The AOTA 
Certification Committee members were consulted and assisted the staff 
in selecting June 21, 1975--one week prior to the administration of the 
NCE. The, committee members feU that this would not give some candidates 
an unfair advantage since all NCE candidates would be invited to' take 
the PE. * 

Many candidates complained- about giving up their last free weekend of 
studying (for the NCE) because of the PE. .Therefore, the staff decided 

to change the sequence of e xaminations so that in January* the PE was 

\ 
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given on the same day but immediately after the NCE was administered. >^ 
The relative proportion of students volunteering, showing up, and com- 
pleting the PE did not- differ as a result of the change in sequence. . 
This led the staff to conclude that some type of incentive to secure 
a larger sample must be considered. It is recommended, that for future ' 
experimental administrations;- subjects should be paid an honorarium. 

Occupational therapy faculty members were asked to voluntarily proctor 
the examination in June 1975. This was viewed as an imposition and 
while many did complete the job, several informed the staff that they 
would not agree to proctor during subsequent administrations. The 
Advisory Committee recommended that the contract be amended so that 
proctors can be paid.. This was approved and all proctors were paid 
an honorarium for the second administration of both levels' of the PE. 

• . t 

The staff had several months to plan for-administration of the assist- 
ant level examinations. It was fairly easy to enlist proctops and 
sites because the program directors were able to select dates when 
^students would be on campus. In fact, many, directors helped/the project 
by requiring their students to participate. The two-part 5O0:-;f-|;em exam-, 
ination was too long for a one-day session, although in some instances - 
that was necessary. ~ 

The second administration for the assistant level examination was 'with- 
out problems. There was just one 250-item examination administered, 
and proctors were paid. 
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FIELD WORK INSTRUMENTS ' 

The only problem encountered with the Qualitative Assessment of Field 
Work Competency (QAFWC) was that of time. The instrument needed to be 
developed and distributed within seven weeks. This resulted in a smaller 
than desirable sample size. Many of the subjects had finished their 
field work by the time it was completed. 

The employer's rating form presented the greatest problem. Subjects' 
interest in the project may have waned, and it was difficult to locate 
them after they left the academic programs. Only 57 of the 123 sub- / 
jects who wrote the PE gave the staff the name and address of their 
employer and only 30 of the 57 employers completed the form. It was, 
therefore, impossible to conduct the follow-up study. 

The project was originally planned for a thirteen-month period. This 
had to be extended for an additional three months because the second 
administrative period of the assistant level PE needed to run through 
"the thirteenth month to obtain the desired sample size. The data was 
collected and analyzed during the fourteenth month. ■ ' 
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RESULTS 



THERAPIST LEVEL 



GENERAL DESCRIPTIVE DATA 

Tables 1 and 2 show the performance of .OTs on the? Proficiency Examina- 
tion (PE), ttie National Certification Examination (NCE) , the Qualitative 
Assessment of Field Work Competency (QAFWC), and be Field Work Perform- 
ahce Report (FWPR). For the Pr^fA^ner^mU^ion, on the average, 
75.14% of the items were answerW cjorrectly by the 123 candidates during 
the June 1975 test administration while 72.65% of the items were answered 
correctly by the 117 candidates sitting for the January 1976 test admin- 
*c istration. •• 

Among the sample of 123 candidates tested during June 1975, 120 of these 
also sat for the NCE. The average percent score for this group on the 
NCE was 69.06. -To check the representativeness of the OT candidate 
volunteers, the score performance of this study population and the total 
number (N=768) of candidates taking the NCE were compared. The average 
percent score for the 768 candidates taking the NCE was found to be 
°67.28. Nonsignificant differences were found between the study volun- 
teers and entry-level OTs for both the total and subarea scores. Inter- 
item consistency reliability coefficients (KR-20) were found to be .87 
, for the PE (N=123) and .88 for the NCE (N=768). 
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Table 1 



Descriptive Data Surory; Proficiency Examination Performance of Occupational Therapists 




Proficiency 
Examination 



Self Care 

,'Vork 

M Play/Leisure 

N Motor Functioning 

i Sensory-Integrative Functioning 

, Cognltlye Functioning 

Psychological Functioning 

Social Functioning 

Life Space 

Total 

' Evaluation 
Planning 
Implementation 



Maximum 
Score 



34 
32 
20 
50 
32 
20 
33 
20 
9 

250 

77 
49 
124 



250-item (N=123) 



Mean 



.26.19 
24.02 
14.20 
37.69 
25.85 
14.59 
24.07 
14.42 
6.81 

187.76 

58.79 
37.46 
91.52 



Standard^ 
Deviation 



2.85 
2.77 
2.23 
4.79 
3.30 
2.75 
3.34 
2.36 
1.53 

17.24 

6.19 
4.25 
8.91 
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Percent 
Correct 



77.02 
75.08 
70.98 
75.38 
80.77 
72.97 
72.95 
72.11 
75.70 

75.14 

76.35 
76.45 
73.81 



Maximum 
Score 



18 
18 
11 
29 
20 
12 
20 
12 
10 

150 

53 
46 
51 



150-item (N=117) 



Mean 



13.32 
13,54 
" 7.68 
21.72 
15.53 
8.90 
13.91 
8.50 
6.63 

108.97 

38,71 
32.84 
36.73 



Standard 
Deviation 



1.79 
2.07 
1.36 
3.31 
2.28 
1.69 
• 2.37 
1.81 
1.85 

13.48 

5.86 
5.09 
5.07 



Percent 
Correct 



'74.00 
75.22 
69.82 
74.90 
77.65 
74.17 
69.55 
70.83 
66.30 

72.65 

73.04 
71.39 
72.02 
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Table 2 

Descriptive Data Summary: National Certification Examination, Qualitative Assessment of Field Work Competency, and FWPR, 
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Instrument 


Maximum 


Mean 


Standard 


' Percent 


• 


Score ■ 




Deviation 


Correct 


National 'Certification Examination (N=120) 

' / 

Biological Sciences. 










32 . 


21.05 


3.96 


65.78. 


Behavioral Sciences 


33 


24.29 


4.06 


73.61 


Clinical Medical-Surgical 


13 


8.59 


2.12 


' 66.08 


Clinical Neurology 


17 


12.10 


2.46 


71.18 


Clinical Orthopedics 


18 


9.95 


2.47 


55.27 




n 


19 16 


JtJX 




O.T. Medical-Surgical 


"18 


13.48 


2.16 


, 74.89. 


O.T, Neurology / 


25 


18.54 


2.46 


74.'l6 


O.T, Orthopedics / ' 


25 


15.76 


"2.63 


- 63.04 


O.T. Psychiatry 


'"42 


29.72 


4.08 


70.76 


Total 


250 , 


172.64 


19.66 


69.06 


Qualitative Assessment of Field Work Competency 








» 


■ (M38J 










Evaluation 


20 


15.32 (3.83) 


'2.69 




Treatment Planning 


10 


7.84 (3.92) 


1.54 




Treatment Implementation 


25 


20.10 (4.02) 


3.56 




Personal Characteristics 


12 


10.08 (4.20) 


2.07 




Total 


67 


58.21 


10:51 




Field Work Performance Report (N*145) / 










Data Gathering 


28 . 


25.58 (3.65) 


2.58 




♦Treatment Planning 


24 


21.16 (3.53) 


2.61 




Treatment Implementation 


80 ' 


71.77 (3.59) 


■6.91 •' 




Communication Skills 


28 


25.47 (3.64) 


2.55 




Professional Characteristics / 


: 56 


49.57 (3.54) 


5.31 ' 




Total • . ' ■ ■ ;•' . 1 


.216 


193.55 


17.92 
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Tables 3 and 4 report the intercorrel anions among the subareas of the 
PE and NCE 1 . Significant moderate correlations were generally found 
among the subareas within each examination. 

Table 5 reports- the intercorrelations between the subareas" of the NCE 
and the subareas of the PE. .With the exception of the Life Space and 
Play/Leisure subareas of the PE and the Clinical Orthopedics subarea 
^ of the NCE, significant moderate correlations were found between the 
two tests. The total scores 'correlation was found to be .61 (p>.001) 
indicating that the two measures share approximately 37% of common 
variance. 

The intercorrelations between the QAFWC and FWPR are reported in 
Table 6. Again, significant moderate correlations have been found 
between these two field "work measures suggesting that there is prob- 
ably a. relationship between the frequency of field work performance as 
measured by the FWPR and the more qualitative aspects of such perform- 
ance measured by the QAFWC. However, these correlations are likely to 
be spuriously high since the same rater used both instruments for each 
subject. Further analysis should be performed by having independent 
rathers for each instrument and sets of raters for each subject. This 
would enable the test user to have available the necessary inter-rater 
reliability data. 

^The sample size of 204 for Table 3 was achieved by standardizing the 
raw scores for both the 250-item and 150-item PE and then combining 
before analyz^g^the^ata^^ Unless specified, all . correlations in this 
report are product moment correTaT^A3~ ;__ 
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Table 3 



Intercorrelaciona Among Subareas 
of the 

Proficiency Examination for Entry Level Occupational Therapists 

(N*204) 



Subereas 



Self Care 
Work 



Self Care Work 



Play/ Motor Sensory Cognitive Psycho- Social Life - 
Leisure Func- Integrative Functioning logical Functioning Space 
tlonlng Functioning 1 Functioning 



.31 



*** 



Play/Leisure 


.04 


,11 


mm — 

t 










Motor Functioning 




,40*** 


.24*** 










i Sensory-Integrative 
^ Functioning 

01 

i Cognitive Functioning 


,34*** 


* .43*** 


.28*** 


.58*** ■ 








,12* 




> 


* *irtrk 
.43 


.48*** 


mmm 




Psychological 4 
Functioning 


.28*** 


',38*** 




.57*** V 


.46 




ttmm 


Social Functioning 


.31*** 


.32*** 


27*** 


.46*** 


.42*** 


.38*** 


,39*** 


Life Space 


,25*** 


.28*** 

V 


.15* 


.33*** 




.31*** 


.38*** .37*** 


Total 


.53*** 


.61*** 


.. .43*** 


J 

.82*** 


.76*** 


.63*** 


.74*** .,66*** 



.53' 



[kick 



. * p ^ .05 
**p<.01 
*** p < .001 
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■ Table 5 

iDtercorrelations Betveen the Subareas of the National Certification Examination and the Subareas of t^Pro?*ic«ncy ExaiinatW 

(fi-120) ! ' 



National Certification 
Examination 



Self 
Care 



Work 



Proficiency Exaaination 
Sensory 

Play Motor Inte- Cognitive Psycho- Social Life 
Leisu: Function grative Function logical Function Space 
Function Function ' 



Totals 



XI 



Biological Sciences 
Behavioral Sciences 
Clinical KedicilrSttrgical 
Clinical Neurology | 
Clinical Orthopedics 
^liolctl Psychiatry 
\T. Medical Surgical 
O.T* Neurology' 
0.1. Orthoptics 
OJ, Paychiatry 

Vocals 



.31*** 
.42*** 

.14 
.22* 
,10 

.27' 



.40 
.24** 

.3* 



32*** 
41*** 

10 
25 1 
14 
32 



34*** 

26** 

25' 
44*** 



e** 



-.10 

.08 
-.07 
-.09 
-.10" 
.00 
.05 
-.01 
.08 
.06 



.44*** .46*** -.01 



.46*** 
.42*** 
.26** 

.31 

.31 



*** 

.41 

-.27*** 
.31 



.39 



*** 



.53 



*** 



.4/ 

.40*** 
** 

** 



,22 
.27 
.18* 



*** 



,28*** 
'.40*** 
.13 

.29' 
.13 



*** 



.37 
.40' 
.27** 



,*M 



.23** ' 
.56*** 
.23** 
.20* 
.06 



.31 
.37 



*** 

*Aft 



.33' 
.27' 
.28*** 
.20* 
.32 



*** 
*** 



.25** 
.30*** 

.14 
.13 

.13 ; 



*** 



.40' 
.34 

K 
.23 

;n*** 



** 
*** 



.52 



.42 



*** 



.48' 



*** 



.23 ; 
.46 1 
.15 
.16" 
,22* 

i 

.34*** 



.08 
.19* 
-.06 
-.05 
.01 

.16 
-.03 
.03 
.16* 

.12 



.44*** 
.57*** 
.22** 
.32*** 
.14 
'.43 



.50 

.31' 
.36' 
.50' 



*** 
m 
*** 
*** 
*** 



.61 



*** 



* p < .05 
*» p < .01 
*** p c .001 
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Table 6 



Interrelations between rhe Qualitative tenement of Field Work Competency end the Field Work Performance Report (FWP R / 

(N«116) 



i 



\ 

\ 



Qualitative Assessment of 
Field Work Competency 






Field Work Performance Report 






Subareas 


uata wicnering 


Treatment 
Planning 


■Treatment 
Implementation 


Communication 
Skills 


Professional 
Characteristics 


Total FWPR 


Evaluation 


.41*** 


.42*** 


■ 1-1. i 

.36*** 


.48*** 


.50*** 


.47*** 


Treatment Planning 


■ .37*** 


.55*** 


.45*** 

; 


.46*** 


.56*** 


.54*** 


Treatment Implementation 


.39*** 


.51*** 


.42*** 


.42*** 


.i-.i_i. 

.51*** 


,50*** 


Personal Characteristics 




.47*** 


.35*** 


^38*** 


.42*** 


.43*** 


Total 


.«*** 


.52*** 


.46*** 


.49*** 


.54*** 


.54*** 


« > < * 

p < .001 


r. 


' \ 

\ 

i 

\ 






< 
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Nonsignificant correlations were predominately found between both field 
work instruments and the PE (see Tables 7 and 8). Even by delineating 
the three process Categories (Evaluation, Planning, and Implementation) 
of the PE, no important significant findings were yielded (see Table 9)., 



Some interesting findings can be observed between the NCE and the field 
work measures (see Tables 10 and 11). Specifically^ the NCE showed a 
nonsignificant correlation (r=.18, N.S.) with the commonly used F.WPR, 
but, a small significant correlation with the QAFWC (r=.28, p>.05). A 
striking finding was that the OT orthopedics subarea of the NCE was 
found to have a moderately strong relationship (r=.57, p>.001) with the 
QAFWC . 

FACTOR ANALYSIS OF THE PE AND THE NCE 

According to the present investigation's scope of work, it was further 
suggested that factor analytic methods be used in the data analysis 
process. The purpose behind their use was: a) to determine whether 
some underlying pattern of relationships or clusters exist such that 
the data could be rearranged and reduced into a set of factors that 
accounted for an appreciable amour, if the oh' rved interrelations in 
the data, and, b) to compare the empirics rii'uings from the factor 
analysis to the original test blueprints of both the PE and the NCE. 

The type of factor analysis performed in the present study consisted 
of a principal components analysis with varlniax rotation of the subarea 
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Table 7 



Intercorrelatlona Between the Qualitative Assessment of Field Work Competency 

and the 

Proficiency Examination for Entry Level Occupational Therapists 

(N=138) 



Qualitative 

Assessment of 

Field Work Competency 






1 : i : 

\ 

Proficiency Examination for Entry Level Occupational Therapists 






Subareas 


Self Care 


Work 

t 


, "ay/. 
Leisure 


Motor 
Func- 
tioning 


Sensory Cognitive Psycho- Social 
Integrative Func- logical Functioning 
Functioning tioning Functioning 


Life 
Space 


Total' -• 


i 

Evaluation 


.10 


.16* 


.02 


.03 


.12 


.02 


.01 


.07 


.07 


.11 


Treatment Planning 


.12 


.16* 


.07 


.09 


.11 


.04 


.06 


.01 


.09 


' .14 


Treatment 
Implementation 


.17* 


.15* 


-.03 


.13 


.12 


.05 . 


.02 


.08 


.10 


.15* 


Personal 
Characteristics 


.07 


-.04 


.01 

! 


.03 


.02 


.05 


-.09 


-.07 , 


.01 


.00 


Total 


.15* 


.10 


.03 


.10 


.09 ' 


.00 


.01 


.06 


.10 


.13 



OJ 

o 



/' 



I 
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?<.05 
p < .01 



Table 8 

Intercorrelations Between th* Field Work .Pekforoance Report (-FWPR) 

and the \ 
Proficiency Examination for Entry Level Occupational Therapists 

(N*145) \ 



Field Work 
Performance Report 



Subareas 



Data Gathering 

Treatment Planning 7 

Treatment 
Implementation 

Communication 
Skills 

Professional 
Characteristics 

Total 



Proficiency Examination for Entry Level Occupational Therapists 

Self Care Work Play/ Motor Sensory Cognitive Psycho- Social Life Total 
Leisure Func " Integrative Func- logical Functioiing Space 
tioning Functioning tioning Functioning 



.08 ■ .08 



.14* 



.1 



.10 .0 

.05 .0 

.12 .0 

.11 .0 



■.07 



.17* 



.10 
.03 



.16* 
.14 



.09 .01 
.06 -.06 



.17* 
.10 



.06 -.03 .10 



.10 -.01 .09 



.02 -.12 .02 



.04 -.14 .04 



.09 



.10 



.00 



.10 



.13 



-.02 -.09 



.03 -.10 



.01 



.05 



.17* 
.13 



.12 



.16* 
.13 



.12 .03 



.07 .03 



.05 
.07' 



* p < .05 
** p i .01 
***p< .001 



Table 9 ! .« 

Interrelations between the Qualitative Assessment of Field Work Competency and the field Work Performance Report (MR) 
, as a function of the three process categories of the Proficiency Examination. 
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Evaluation 


Proficiency Exacination 
Planning 


Implementation 


Qualitative Assessment of 
Field Work Competency 

(N=13S) 

Evaluation 
Treatment Planning 
Treatment Implementation 
Personal Characteristics 


.10 
.12 
.15* 
.02 


.05 

1 ) 

. A k 

. .13 
-.04 


,10 . 
.13 

,13 / 
M 


Total 


.13 


.11 


,10 


Field Work Performance 
Repon; 

(w«i 

Data Gathering 
Treatment Planning 
/Treatment Implementation 
Communication Skills 
Professional Characteristics 


.13 
.12 
-.01 

.02 
.■I 


.16 
.11 
.05 

.01 


.13 

. .11 \ 
.01 

■ .02 
.04 


Total 


.04 


.12 


.05 

i 



52 



* P < .05 
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Table 11 



iMeriWriatlOBB between Field Work Perforate Report and the National CertlM cation Exanination 



National Certification Examination 



w 



" Fieic. tutr'n 
Perforcanc? !vn :, 



J.JfSlo- Behav- CilakaK Clinical Clinical Clinical .O.T. O.T. OT OT 
[•logical lor«l Medical- ftur 1 w Ortho- Peych. Medical- Neurology Ortho- Psjch. 



Pitt Ciiherlog 
Trtacaent Plaaalag 
Treataent foplctttotitloo 
Cescwlcfltioo Ski lit 
Professional Character . 
iatici 

Total 



Scienceo Sciences Surgical 1 



pedlcR 



Surgical 



.16 
,17 
.08 
.10 
.19 



..03 
.02 
-.09 
-.05 
-.11 



.20 -.03 



.09 
.08 
-.04 
-,06 
•,00 



/ 

.19 



.0/ 

.24* 

.19 

.18 

.20 

■ .29* 



.04 
.08 
.06 
.04 
.04 

,01 



.10 
.08 
.05 
.06 
.04 

.05 



.31** 

•16 
.01 

.14 
.21 

.1? 



-.10 
-.05 
-.02 
-,06 
.09 

-.04 



pedfcs 



.04 
.16 
,06 
.08 
.14 

;24: 



Total 



.31** 

,29* 

.01 

.06 

,09 

.19 



.17 
.19 

.05 
.07 
.11 



61 
58 
50 
62 
44 



.18 35 



5 



* P * .05 
*« P < .01 . 
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scores on the PE and NCE. Each factor or component consists of and is 
defined by those variables loading highest on the factor. Table 12 
reports the final factor analytic solution for the PE's nine subareas. 
Four orthogonal (uncorrelated) factors were extracted accounting for 73% 
of the total variance. It can be observed by examining the proportion 
of the total variance accounted for by each specific factor that 42% 
of the variance was explained by Factor I. These findings suggest that 
the predominant characteristic of the PE is defined by Factor I. 

To identify the nature of each of the factors, one must examine the load- 
ings of each subarea on each factor. These loadings are simply correla- 
tions between each subarea and the factor. Below Factor I on Table 12, 
it can be observed that the subareas of Motor Functioning, Sensory-Inte- 
grative Functioning, Cognitive Functioning, and Social Functioning have 
the highest loadings on that factor, that is these subareas tend to 
cluster. The Advisory Commit- continued to examine the remaining 
factor loadings on each of the four factors, and attempted to identify 
the structure of each factor and the total examination. The four factors 
for the PE were identified and labeled as follows: 



O ( 



- 35 



Table 12 

Rotated Factors (Varimax) for the Proficiency Examination Subareas 

(M23) ■ 

* ,1 









Factor Loadings 




Variable 


I r 


II ■ 


III 


^ IV 


Self Care' 


,43 


.44 


.33 


-.38 


Work 


.11 ' 


.12 


.50 


.12 


Play/Leisure 


.12 


.12 


.10 


.89 


Motor Functioning 


.73 


.15 


.41 


-.10 


Sensory-Integrative Functioning 


.62 


.11 


.56 


.07 


Cognitive Functioning 


.65 


^.07 


.26 


,41 


Psychological Functioning 


.41 


.47 


•• .27 


.32 


Social Functioning 


.82 


.19 


-.09 


.12 ■ 


Life Space 


• .10 


.90 


.06 


.09 


Eigenvalue 


3.78 


1.13 


.88 


.78 Total Var. 


Percent of Total Variance 


42.00 


12.50 


9.70 


' ; 8.70 73.00 
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- t Proficiency Examination 

' ' : 

Factor I Social -Motor Function 

Factor II Life-Space 

Factor II Work 

Factor IV Play/Leisure 

It should be pointed out that a number of the subareas showed moderate 
correlations on more than one factor. For example, the Sensory-Inte- 
grative Functioning 5 subarea loaded .62 on Factor I and .56 on Factor III 
This, subtest would be considered to be "factorially complex" and most 
likely composed of items that are characteristic of both factors-. 

Although not shown, the three process categories of Evaluation, Plan- 
ning, and Implementation, which cut across the nine subareas, were also 
factor analyzed. These three processes of OT functioning were not inde- 
pendent as indicated by their approximately equal loadings on only one 
factor. The use of the process categories may have some utility for a 
test's descriotion and as a classification scheme for delineating OT 
tasks, but candidates' responses do not show differing patterns. 

Table 13 shows the varimax rotated factors for the NCE. The subareas 
ofBehavioral Sciences, Clinical Psychiatry, and OT Psychiatry, compose 
the "general" factor of the examination with 41.5% of the total variance 
explained by that factor. The Advisory Comrrittee examined the loading 
and provided the following interpretive labels for the four orthogonal 
factors.\ 



Table 13 



Rotated Factors (Varimax) for the National Certification Examination Subareas 

(H20) 

I . i 



Variable 


I 


Factor Loa 

11 ' 


dings 

III 


IV 


Biological Sciences 
Behavioral Sciences 
Clinical Medical-Surgical 
Clinical Neurology 
Clinical Orthopedics 
Clinical Psychiatry' 
0.1, Medical Surgical 
0.T, Neurology 
0,1, Orthopedics 
O.I. Psychiatry 


,24 ■ 

.82 

.29 

.27 

.01 

.82 

.26 ,. 

.13 

.10 

. .74 


.67 
.15 
ijj 

.65 * 

.80 

.34 

.02 

.22 

.30 

.04 


.37 
.29 
.31 
.06 
.02 
.00 
.78 
.13 
"".71 
.33 


.17 
.14 ' 
.08 
.37 

.04 ' 
-.05 
,12 

,92 ' 

.05 

.25 


Eigenvalue 

Percent of Total Variance 


4,15 , 
41.50 


I. 19 

II. 90 


.85 
8.50 


.16 Total Var.i 
7.60 69.40 
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Factor I 
Factor II 
Factor III 
Factor IV 



National Certification Examination 

Behavioral Sciences/Clinical Psychology 

Orthopedics 

Medical/Surgical 

OT Neurology 



Comparing the PE with the NCE, six out of nine subareas (representing 1 
189 items) of the PE had moderate to high ' loadings on the first general 
factor, while only three out>of ten of the subareas of the NCE (repre- 
senting 102 items) loaded substantially on the firsv factor. 

A factor analysis of the combined subareas of the PE and NCE was per- 
formed to provide a general description of the field of occupational 
therapy. Four factors were generated again since factoring beyond this 
point did not yield factors with, any appreciable amount of explained 
variance. Table 14 shows that Factor I was composed of those subareas 
which also load highly on Factor I of the PE (Table 12). The OT Medical/ 
Surgical subarea of the NCE was the only subarea to load moderate > on 
this first factor. The first factor. that emerged (Table 13) for the 
NCE is shown under Factor III and is accompanied by the. moderate load- 
ing of the Psychological Functioning subarea of the PE. . We may conclude 
from this, that the general factor, of the PE is explaining a greater pro- 
portion of the combined test variance than is the first factor of the NCE. 
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Table It 



Rotated Factors (Varhax) for the National Certification And the Proficiency Exaction Subareas Coined 

(N-120) 



O 



Variable 



Biological Sciences 
Behavioral Sciences 
Clinical Hedicsl-Surgical 
Clinical Neurology 
Clinical Orthopedics 
Clinic*! Psychiatry 
O.T, M:sl Surgical 
O.T. Neurology 
O.T. Orthopedics 
0,1. Psychiatry 



Self care 
Work 

Play/Leisure 
Motor Functioning 
Seosory-Integratlve Functioning 
Cognitive Functioning 
Psychological Functioning 
Social Functtetng 
Life- Space 

Eigenvalue 

Percent of (oral variance 



.32 

.27 

.02 

05 

.05 

.10 

.60. 

.09 

.28 

.22 

.69 

.35 

.01 

.67 

.63 

.40 

.36 

.67, 

.55 

6..0 
33.70 



II 



.72 
.23 
.56 
,7V 
.67 
.29 
.21 
.58 
.40 
.21 
.10 
.19 

-.18 
.32 
.32 
.27 

-.02 
.11 

-.27 

2.07 
10.90 



Factor Loadings 



III 



.22 
.80 
.38 
.24 

-.02, 
.70 
.. .29 
.17 
.30 
.76 
,29 
.37 

,.09 
.20 
.18 
.15 
.60 
.03 
.15 

1.15 
6.10 



IV 



-.05 
.09 

».U 
.06 



02- 



.07 
-.07 
.15 
-,05. 
.06 
-.19 
.29 
.82 
.22 
,3? 
.66 
.38 
.28. 
.15 

1.09 
5,80 



Toul 
Variance: 
56.50 
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she Advisory Coanutee reccsranded that c*rwj?<iatev qafwc and F«P3 
scor.-.. ce compared across the psychiatric * fH} physical dH*biJuy 
settings. Scores fro?! $6 candidates en the QA?UC anO S3 candidates 
on the FWfc *er* ^ur^ed for each student under each setting. Kith 

eACepUon of tte ^ta Gathering suh3*ea of the raPR, the within 
subject analysis of variance reported rn table IS did not yield any 
stgntf icant differences, 'in the perforce an either *ns»ru**ftt at- 
their two settings. ■ , 

/ 

Ctirricuiun directors ^re asied, based upon the Advisor;, Caen* wee's 
reccntsendation, so provide the Grade s»o»«i Averages fGPA) for those 
candidates who voSgnteered for the January testing of the PC. 7b<< 
Misor. of wadmtc achievement .as then correlated, ^uh the PC , the 
9»"WC, and the FKPR. Table 4 resorts tnat four of the sutereat of 
the P£ and i«o swoareas and iou! scores of the QAF«C shc**e nqnift- 
cant correlations with GWt. The WR did not s**. m y ^Utcwi 
relationship with 



ASSISTANT Itm 



H ! Wl OSSCR IPTtVt DATA 



* l0Ul 0f nv ' completed the occupational therapy assistant 

level form of the Pi, Descriptive s^ry data is reported in Table 17. 
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PAychaiagu-*! hnaUr^n^ 
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Pain? to' 
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An average of 72,88* of the items were correctly answered by this 
population. The most difficult subarea of the ?E was Psychological 
functioning (67.09%), and the least difficult was Self Care (76'. 99%). 
The Kuder/Richardson 20 re natality for the P£ was calculated and 
found to be .87. 

Out of the 173 OTA* tested during the second test administration, UO 
OTAs ^ad notched scores on the Qualitative Assessment of field Worfc 
Performance (QAFWP). Table IB reports the intercorrelattons between 
the OAfWP and the assistant \<tvtt\ Law-order sig^i f ic«tnt correla- 
tions are noted between the Treatment rlanninq and Treatment i^^wt-'" 
atum -s^ibareas of the^AFw? and the work subtest, Piav/ieisure subtest-, 
ana total scores of the assntant 5eve' ?£. The Evaluation Process 
wbar^a of t:*if Pi showed overaH $s*-order r^aro. f Kant correlation with 
ife OAFWP.'" 

•prif.c 3 co«A"i3r<*it < . 'f\K-tor ana^^ts wsift '.=arvr\ ; $A rotation ^as pe«** 
fcnric-d i^e sub*r*at of th# asv^Stant krvel Pi (Tab^e W Pour 
factors *c*rt generated accounting for ?'j SO* of the tata^ - :rv ,k s cv . A 
he--K> '"actor lead t n-': { A-ik) *.-y-, found to? the Wo'sr ?y**ct subare.-* 
?>f tit*; PS cfi Factor i Although a .?v^^r of the :.<jb tests t'n* Pi 

as \ £ V<> *f-r f .»r :r;? M ". tv thtf A,d rUor/ Cxr : t T-V* . . 

■ 7u 



T*blc li 

Descriptive' D«r.A Suirary 
Occypiktlor.il Therapy Aishtint Proficiency E*ifllaitlor. 



OTA ?wslcl«cy Exr.ioislw J swiw 

i Score 



Averse fUv 

Store?- 



T«*l 

Phy/Lf!a«rt 
lift Jpitt 



250 

♦ ■ 



1M 

MJ 
MS 

!»:: 

M? 



ISM'' 
Jl.H 



1 



k' 1 



P«rcw 
Cwreci 



74,4? 
■4M'< 



Deviation 
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Table 18 

Incercorrelations Between the Qualitative Assessment for OTA'- md the 

Proficiency Exulnatlon for Entry level Occupational Therapisi- siistantj 

(N40) 



< 

QwUtatta 


Proficiency Examination for Entry Level Occupation! Therapist 






1 








Self 

Care U'ork 


Play/ 
leisu'e 


Motor 
Func- 
tioning 


Cognitive 
Func- 
tioning 


Psycho- 
logical 
Func- 
tioning 


Sock 1 
Font, 
tlonli^ 


We 




Eva his i ion 


Planning Imple- 
mentation 


Evaluation 


.0? .13 




.10 


.01 


.08 


.04 


,01 


12 


.1-4* 


' ,05 


.0') 


!re»tfient Pitfinlng 


.10 if 


D 

.IB 




f i 


.13 


.Of 




,21** 


*i 

.23 


.12 


a 

.16 


Tf««ts«nt 


.21** 




.15' 


1 t 
* * * 


.06 


.04 


■;? 


* 

,19 


M H 


.00 




Hole Expwtafbns 


.0' ,0'; 




-.01 


.11 


,04 




0.00 


.0? 


,!</ 


,0! 


.04 


Tots' 1 


.0? ,12 


.12 


.01 


,01 






.« 


.10 


.1? 


.0) 


.0? 



n 



* /.os 
*» p< ,ji 
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Tabic 19 



Rotated Factors (Varimax) for the OTA Proficiency Examination Subareas 



Variable 


I 


Factor 
II 


Loadings 
ill. 


IV 


Self Care 


.61 


.OB 


.54 


.14 


Work 


.51 


JO 


.5; 


.12 


PUy/UUure 


.;) 


,46 




.11 


Motor funct ioning 


.86 


.12 


.1)4 


.05 


Cognitive Functioning 


.08 


. 14 


.30 


.18 


Psychological Functioning 


-.33 


. o 


.4? 




SocUl Functioning 


,36 


.73 


.04 


.3) 


Ufe Space 




.14 


. M 


• '0 


Eigenvalue 


3,56 


.St. 


..'7 


.t? 


Percent 1 *! Total Variance 


• * a 


10. .10 




8.40 



Tot«i Variance » ?J.Y? 



Factor I Motor Functioning 

Factor II Psycho-Social Functioning 

Factor III Cognitive Functioning 

Factor IV Life Space 

TH£_COMPA RATIVE SCORE PERFORMANCE BET j^EN L QTs_AW OTAs JN THE THERAPIST 
LEVEL PROFICIENCY EXAMINATIO N 

123 therapist level scores and 119 assistant level scores were oboined 
from OTA candidates writing the therapist level PE, Table 20 reports 
the comparative performance between these two groups for total and sub- 
area scores of the therapist level PE. labia 21 reports a discriminant 
anal/sis of the same two examinations. Frequency distributions of the 
scores obtained from each group are also provided in Figures 1 through 10, 

'ha general finding was that OTs performed significantly higher than OTAs 
on the therapist level PE.. it was also observed \hat . the therapist total 
score distribution war, negatively skewed aM leptokurtic (see Figure 1). 
The skewness of the distribution is predominantly a function of a diffi- 

t 

culty of tno ite-$ white *m- lootokurttc shape is &,«? primarily to the 
reduced variance 6f the set of scores. This type of distribution *< 
often cited to be a Characters stic of the cruerton-r«fere<tc«<l test. The 
steep slope on the accelerating .;urve is also advantageous when attempt- 
ing to partition candidates into Mastery- and "non-rastcry" categories 
witn Httle assignment error The OTAs * score-, appear wr* nn^riy dtv- 
touted alo=ig the total score PX ihnenstcn 

7 n 
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Frequency Distributions of Scores for Occupational Therapists and Occupational Therapy Assistants by Suoarea: 

Psychological Functioning 

Figure 8 
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Frequency Distributions of Scores for Occupational Therapists and Occupational Therapy Assistants by Subarea 

- Social Functioning 

I 

Figure 9 . 
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Frequency Distributions of Scores for Occupational Therapists and Occupational Therapy Assistants by Subarea 

Life Space 

Figure 10 
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According to the data presented in Table 3, the various subareas of 
the PE show significant moderate intercorrel ations. Bast d upon this, 
it was not surprising co find '.)Ts performing significantly higher than 
OTAs on all the subtests. Because of this interrelationship, it was 
decided to explore, through discriminate analysis, which particular 
subtests of the therapist level PE maximally differentiates OTs from '< 
OTAs. This analysis is reported in Table 21 and indicates that the 
Motor Functioning subtest is the dominant subtest that maximally dis- 
criminates the two populations. To a lesser degree, the Sensory- 
Integrative Functioning and the Cognitive Functioning subtest also 
added significantly in stepwise fashion to this differentiation. 

o 
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mm and fKaiwvrioNs 

v The intended purpose of the present investigation was "to evaluate 
«• 

and analyze the reliability and validity of measurements that are 

•' / .... 

predictive of competency and proficiency at entry levels in occupa- 
tional therapy." The data collected and analyzed from this study 
yields the following overall pattern. 

THERAPIST LEVCL 

Therapist level PE scores showed a significant moderate correlation 
with NCE scores^ (r=. 61 , pxOOl) indicating a moderate correspondence 
between the criterion-referenced PE and the norm- referenced NCE. The 
probability of finding a significant relationship of this magnitude 
would be favorably based upon the fact that each measure was in part 
intended to survey "achievement" in the field of occupational therapy. 

A number of divergencies ware also noted between the PE and the NCE. 
In particular, the Play/Leisure and Life Space subareas of the PE and 
the Clinical Orthopedics subarea of the NCE were not among the various 
significant correlations observed between these two measures. A com- 
monality among these subareas is that each was independent of the first 
factor found for that examination. Specifically, Life Space was identi- 
fied as Factor II and PTay/Leisure^s Factor IV for the PE factor solu- 
tion reported in Table 12. Clinical Orthopedics was identified loading 
heavily on Factor II for the NCE factor solution shown in Table 13. 

102 
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The factor analytic data produced a number of pertinent findings. The 

Advisory Committee was able to determine, . by examining >the loadings of 

each subarea on each factor, the composition of the major components^ 

each examination. Although approximately 40% of the variance in total 

test scores could be explained by the first factor for each examination 

two-thirds of the subareas on the PE and approximately one-third of the 
v 

subareas on the NCE I jd moderate ;o high loadings on the first factor. 

This could suggest a more homogenous set of content is being measured 

• by the'PE, Given the greater homogeneity of the PE, it was not sur- * 

prising to observe Factor I of the PE explaining a greater proportion 

of the combined tests variance than Factor I of the NCE. 
./ • • 

Nonsignificant correlations were generally observed between both the 

FWPR aruLQAFWC with the PE. Although the. relationship for total scores 
between, the PE and QAFWC (r=13) would be significant at p>.065, these 
two measures share less than 2% in common variance. The NCE also demon- 
strated a nonsignificant relationship with the FWPR (r=..18, NS), but. 
fared better with the QAFWC (r=.28, p?.05). External rating instru- 
ments (i.e. supervisor's ratings) rarely show greater than low-order 
correlations with written examinations. Therefore, further research 
designed to determine what, kind of performance is being measured by the 
field work instruments, and not presently measured by the PE and NCE, 
should be undertaken. Not performed in the present investigation but 
a necessary ingredient for further validation efforts, is the determi- 
nation of the inter-rater reliabilities of the FWPR ; and QAFWC. 
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The general finding that candidates' QAFWC and FWPR scores did not 
differ significantly across their field work placements provides 
empirical support that the instruments are not biased in favor of any 
specific field activity, but provide a measure of general field work 

A significant relationship between an academic achievement measure 
(GPA) and scores-de rived from the PE and QAFWC was found.. The signi- 
ficant predictive validity coefficients found for the QAFWC and 7 not - 
for the FWPR, supports the need for continued development of a quali- 
tative instrument. ^ 

ASSISTANT LEVEL 

/ 

The Treatment Planning and Treatment Implementation subareas of the 
QAFWP showed a significant low-order relationship with total PE scores. 
Although only 20% of the items on the PE are concerned with the evalua- 
tion process, this variable of the PE showed significant findings 
throughout the scales of the QAFWP. This implies that if OTAs are able 
to evalute OT problems and situations effectively, shown on a writ- 
ten examination, this ability is also related to receiving higher per- 
formance ratings, 

! 

The Motor Functioning subarea dominated Factor I of the assistant 

PE, whereas the Motor and Social Functioning subareas have the highest 

loading on Factor I. A Life Space factor was consistently found for 
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both levels of the P£. The five sobareas of the assistant level Pi 
were also found to load on more than one factor suggesting that with 
the exception of the Motor functioning, Cognitive functioning, and 
Life Space subareas, the remaining subtests are not composed of a 
homogeneous set of items. If items with known factorial composition 
are placed within each subtest, comparisons with otHer measures per- 
mit clearer interpretation. 

The .comparison of OTs score performance with that ( of OTws on the hera 
pist level P£, demonstrated that OTs performed significantly higher" 
than assistants and that the scores obtained on the Motor Functioning 
subtest, in particular, maximally differentiate these two populations. 

RECOMMENDATIONS FOR FURTHER RESEARCH 

1) S .ore perfcmnce between graduates from approved AGTA programs and 
individuals not having such education should be compared. 

s '7/ 

2) A measure of assessing the on-the-job performance /of individuals 

I 

eligible for proficiency entry into the field shouwj be developed and 
validated against criteria designed to measure comj^tency *\ e.g. prof i- 
ciertcy examinations). 'v, 

3) A follow-up investigation of how well these e&y-Tevel measures 
predict future job performance should be conducted. 



The relationship won§ subareas found through the factor -analytic' 
studies differed from the judgements made aboyt the practice ofHaccu--- 
pational therapy, For example, four subareas of the therapist level ■' 
P£ clustered together on one factor indicating that they are measuring 
common variance. 

The task inventories were designed by a panel of experts, efforts 
were made to empirically study the inventories, so it is difficult to 
view them with assurance 'as\defin1tfve-. statements of the practice of 

V 

occupational therapy. We know from the task analysis that different 
categorizations of subareas may exist. We. also, know, from reports of 
item writers and test construction consultants, that it was difficult 
to assign items to categories and that overlapping of subject and 
process was a problem. This supports the conclusion that the test 
blueprints (designed from the task inventories) were not an ideal frame- 
work for the PEs. 

Methodology consisting only of expert judgement; raises another p<oblem. 
The resultant examination cannot be .systematically and consistently 
updated. The project staff recorownds that a procedure of identifica- 
tion or verification of occupational therapy practice ^ conducted. It 
should be based upon the collection of data from a large sample of 
practitioners. A standard method of data collection could "be repeated 
to update examinations on a regular basis. 
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The project staff further recoircnends that in future studies of occu- 
pational tfcerapy role, identification, "the data obtained b;< factor 
analysis in this project be considered. The four factors identified 
for each level e<a nnation (see pages-37 and 48} may be considered 
as- content subareas. 
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' FUTURE-USE OF Tf£ PROFICIENCY AT© 
•* . V." PROPOSED USE OF THE PERFORMANCE INSTALMENTS 

ASSISTANT LEVa PROFICIENCY EXAMINATIONS 

• / 

In October 1975,, the policy-making body of The American Occupational 
Therapy Association (AOTA), the Delegate Assembly, resolved that a 
written examination would be a requirement for certification of occu- 
pational therapy assistants. (Ttfe previous requirements were the com- 
pletion of an approved program andfield work experiences and the recom- 
mendation of, the program director.) In April 1976, following the delib- 
erations of a special task force and receipt o'l the permisVioK^f the 
staff of Department of Health, Education, and Welfare, Diyisiui of J 
Associated Health Professions, the Delegate Assembly resolved to u!e 
the 250 items from the assistant level proficiency examination item 
pool as a certification examination. These 250 items are to oe 
selected to conform with the blueprint of the PE (see Appendix D ) . 
The examination will be first administered on June 25,- 1977, to all ' 
graduates of approved programs completing all requirements after 
October 31 , 1976. T'ie examination will be administered, twice annually 
beginning in 1978. The AOTA will, be responsible for updating and revi" 
sion of the examination after it's first administration in .1977. 

RATERS' EVALUATION OF FIELD WORK MEASURES \ 

■- v ; - ' " 

A rater's evaluation was also distributed to all supervisors along'with 
the Qualitative Assessment of Field Work Competency (QAFWC). 251 raters 



evaluations were returned. (A copy of tjie evaluation form and the 
data summary are in Appendix H.) There were several suggestions for 
revision, but most of the results indicated that the instrument was 
viewed as valuable. This information, coupled with some dissatisfac- 
tion with the Field Work Performance Report (FWPR), has led the 
Advisory Committee to recommend that the project staff propose the 
use of the QAFWC to the AOTA. The project staff will suggest to the 
Association that a committee. be formed to study this possibility. 

A rater's evaluation was also distributed to all supervisors along with 
the QAFWP. 170 raters' evaluations were returned from 170 different 
raters. (A copy of the evaluation form and the data summary are in 
Appendix' H.) Again, the response was favorable. It should be pointed 
out that currently there' is no standard and/or official measure of field 
work performance at the assistant level . Each school uses a different 
instrument. The occupational therapy assistant program directors, 
faculty, and field work supervisors all have expressed a need for an 
official instrument and felt that the QAFWP would be more than adequate. 
The Advisory Committee recommended tha/the project staff propose that 
the AOTA consider studying 'the QAFWP for its adoption as an official 
instrument of field work performance. The project staff will -propose 
.to the AOTA that .the SaFWP be used experimentally for a one-year period 
during which time eaclil schoo] would continue using the form that they ' 
currently use. A committee should study and design an experiment and 
then present its recommendations to AOTA. 
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tasks _2_ questions 



tas ks l_ questions 



tasks J_ questions 




24 questions 



12_ tasks 52 questions 



22__ tasks 55_ questions 



12_ tasks 35_ questions 



tasks 9 questions 



J_ tasks 7 questions 



_2_ tasks questions 



3 tasks 9 Questions 



JK tasks J)_ questions 



174 questions 



J- 



73 



74 



49 



13 



11 



11 



25C Q 
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APPENDIX E 
FWPR 



AMERICAN OCCUPATIONAL/THERAPY ASSOCIATION, INC. 



RATER'S GUIDE 



fieTld work performance report 



purpose 

Field work is an important aspect of the student's professional education. This 
is his opportunity to gain experience under supervision in the application of his 
didactic training. Since field work is part of the total educational process, 
strengths an] «uiggestions for improvement should be stressed in order to promote 
professional gro wth in the student. Successful completion of all field work is 
required for admission to the certification examination of the American Occu- 
pational Therapy Association. it is essential that each student be rated on a 
standard form in accordance with a uniform rating procedure in order to (1) pro- 
vide the curriculum director with a complete picture of the adequacy of the stu- 
dent's preparation to practice occupational therapy and (2) enable the curriculum ' 
director to recommend the student to write the certi f ication examination. 

This form has been designed as a counseling tool as well as for evaluation; there- 
fore, it should be used throughout the field experience, especially at mid-term. 

GENERAL PRINCIPLES v 

To obtain an accurate appraisal of a student's performance, the following principles 
should be observed: » 

\ 

Consider each student separately since no student performs equally in all aspects 
of his behavior. 

Observation of instances cf specific behavior rather than general impressions is 
fund,.j rental to objective evaluation. 

If i. student's performance has markedly improved during the latter half of the 
field experience and it^jy* Jelt that this performance is mor* typical, utilize 
that period as the basi^Wor the final evaluation. 

ADMINISTRATION / 1 

All the required entries on the top of page 1 of the ' form must be completed 

If a st-.adent works under the direction of more than one therapist, the evaluation 

should represent their combined judgment , even though there is but one siqnature 

required. 

Check marks, rather than numerical scores, should be entered on the FWPR form 
Place check marks within the appropriate box and not on the line between columns 
tor <:he final evaluation. 

The Scorer's Guide is primarily for the use of the academic supervisor but is made \ 
available to the field supervisor (rater) for evaluation of marginal performance. - ) 
If rarely" or occasionally" columns are used for the majority of items, the 
rater should score the center's copy so that the student may be informed of border- 
line or failing status. ' .. 

If a- student completing the last field work assignment the day before writing the 
Certification (Registration) Examination is failing, the academic supervisor must 
be notified by telephone at least .three days before the- date of the examination 
in order that admission to the examination may be cancelled. 



Rev. 3/74 
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Students who request a copy of their FWPR may be allowed to duplicate a copy- 
One copy (or more if so requested) should be returned to the student's academic 
supervisor IMMEDIATELY upon completion of the field work experience. 



RATER* S GUIDE 

This form is divided into five sections: 

Data Gathering 
Treatment Planning 
Treatment Implementation 
Communication skills 
Professional Characteristics 

Under each section heading are statements of various aspects of behavior possible. 
The qualitative aspects of required performance are inherent in each statement. Each 
statement is to be considered specifically in terms of the section it ,is delineating. 

' * •* . • * ' 

The behavior statement should be counted as having been performed by the student 
only if it was performed independently with the amount of supervision normally given 
students at your facility. 

For each statement, check the definition that best applies to the student's be- 
havior as observed. The following definitions of the adverbial headings are to 
be used. 



Cons i stent ly : 
Frequently: 
Occasionally: 
Rarely: 



Given 10 opportunities, the student displayed this behavior 8, 9, or 
10 times. (More than 75% of opportunities) 

Given 10 opportunities, the student displayed this behavior 5, 6, or 
1 , times. (50-75% of opportunities) 

Given 10 opportunities, the student displayed this behavior 3 or 4 
•times. (25-50% of opportunities) 

Given 10 opportunities, the student displayed this behavior 2 times 
-/ or less. {Less than 25% of opportunities) 

If a particular statement is not observable at your facility, or was not observed 
in the student, leave it blank and enter the letters N.O. (Not Observed) in the 
Comments column. The student will in no way be penalized for an N.O. entry. Ex- 
tensive use of the N.O. category, however, indicates the need to re-examine your 
interpretation of the statements. If you send in a form with more than twelve (12) 
N.O.'s, it will be returned to you for re-evaluation. - 

An entry should be made for every statement, 
not be made on a line between the columns. 



Entries for final evaluations may 



ADDITIONAL COMMENTS SECTION > 

Comments' section might include: nature of caseload; notable strengths and weak- 
nesses; potential for work in this area. 

This section is also to be used for any further information or explanations deemed 
nelprul to the student or the curriculum. 



SIGNATURES 

After entering the check marks on the entire evaluation form, a thorough explanation 
should be given to the student at his final conference. Both the supervisor's and ' 
the student's signatures should be entered on the form, only after the report has i 
been fully discussed. Please notejihat the student's signature denotes only that 
he has read the report and does not imply that he agrees with it. 
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GUIDELINES 



Data Gathering 

This section covers all information necessary to treatment or program plan- 
ning. The information or methods of gathering will vary considerably depending 
upon the setting, (e.g.: background information from charts, admission confer- 
ences, family interviews, referring agencies, observations, and evaluative pro- 
cedures.) 

Suggested considerations for individual statements: 

1. Pretest information means ajny information needed prior to evaluating, 
not exclusively "testing". 

2. includes both the choice D f the proper evaluation method, from information 
gathered in item 1. and choice of the proper tools and materials for eval- 
uation. 

3. "Climate" refers to all aspects of the situation. surrounding the patient/ 
client, (e.g.: crowding, noise level, temperature, seating arrangement, 
stress from family members or others.) 

4. Note: "standardized or recommended technique", not necessarily both. 

5. While this opportunity does not arise in the field work experience as fre- 
quently as other items in data gathering, it is valuable as an indicator 
of total comprehension of the situation and/or creative thinking on the 
student's part. Consider recognition of the need to adapt as well as the 
ability to adapt. 

7. Includes three different areas but it is important that therapists consider 
all equally. if a student places unequal emphasis, this should be mentioned 
in the comments section. 

Treatment Planning v 

Results from data gathering are used for treatment or program planning in colla- 
boration with staff and others, including the patient/client when appropriate. The 
extent of program planning varies considerably depending upon the setting and the 
resources available. 

Suggested considerations for individual statements: 

8. Emphasis is on "define problem areas " through knowledgeable interpretation 
of evaluative procedure previously used. 

9. in many situations this is a hard item to. implement, e.g., long term ca're 
where discharge is improbable; crisis situation where discharge appears pre- 
mature it 1S essential, however, that /these factors be recognized and con- 
sidered in whatever- is planned. 

10. & 11. Assume it is done when and if appropriate. Also, consider collaboration 

with the family when it is important. 
13. Note "or" in this statement. Alternatives are almost always available thouqh 

not necessarily innovative. 

Treatment Implementation 

in rating this section consider the student's ability to provide the needed environ- 
ment, tentunate treatment, be aware of group dynamics, be a part of the team approach 
and use community resources. FF 
Suggested considerations for individual statements: 

16. Structuring the activity area or providing the opportunity for "the patient/ 
client to choose materials and develop own work space 

17. Seating patient/client (with a group or isolated); table height; bed posi- 
tion; distracting elements. ' p 

22. "Environment" encompasses such factors as treating the patient/client at bed-' 
81de *; n clinic, alone or as part of a group, in an outside recreation area, 
or with family present. 

28. Refers ^terminating treatment in broad sense; e.g., at discharge; when max- 
imum benefits have been reached; when student permanently leaves the settim • 
JowSeilitJ*) treatment session; (Whatever pertains to the expectations- of 

30 ' Br f f^° the a ,^igned caseload; patients/clients in one specific group, and/ 

or total group physically present in a setting. 
31. involving others in providing a consistent approach for maximum support and' 

reinforcement in achieving goals. 

32> Sf^d'barrlers'whii 38 * COnnotati ° n and is -»»* to include both bene- 

' or e «on * relation t0 th « home, school, any agency 
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Communication Skills ' \ 

; \ 

Communication is used in a broad sense. It is a major factor in projecting 
the image of occupational therapy. Obviously it must be effective and this 
quality is inherent in each statement. It is always viewed as a two-way pro- 
cess including;-; both input and output. /Speaking and listening or writing and 
reading are of/ equal importance. 

Suggested considerations for individual statements: 

38. & 39. "Appropriate form, content, and manner" or "according to require- 
ments of; the facility" will vary considerably with setting. This should 
be clarified during orientation. 

Professional^ Characteristics 

This section addresses itself to those attitudes and attributes that enable a 
person to perform at a consistently mature and professional level. Some of 
the characteristics are administrative in nature. 
Suggested considerations for individual statements : 

41 m Sets priorities for work load; organizes self to accomplish necessary 
tasks. 

42. Returns equipment to proper location? shares responsibility for clean- 
up of area, maintenance and repair of equipment, neatness of work and 
office space. * 

43. Notifies proper personnel of supply needs; suggests new items. 

44. Takes as much responsibility for all aspects of patient/client treat- 
ment as is expected at the facility; maintains contact with other team 
members; displays initiative in crisis situations (e.g.: taking over 
for ill staff member.) 

45. Alters work pace to adjust to day-to-day changes J^-ro'utine and work 
load; finds alternative activity if program is'temporarily slack. 

46. Takes initiative to seek information; uses discretion in wording and 
- • timing of inquiries. 

47. Grows professionally in response to dialogue with and observation of 
supervisor and staff. 

49. Discusses problems with supervisor or other appropriate persons, re- 
quests medical or counseling service if needed; requests change in y 
amount of supervision, nature of casajoad, amount of work; gives con- 
structive suggestions concerning departmental policies', procedures, 
staff, etc,; brings personal problems to the attention of staff or 
supervisor when indicated (e.g.: health*, finances.) 

51. "Other areas' 1 might include: community agencies, local political situ- 
ations, nutritional aspects, architectural barriers, financial impli- 
cations, r legislative concerns. 

52. Takes initiative to independently arrange or seek out: observations, 
field trips, additional experiences iiwi.de or outside the facility, 
reading material, voluntary attendance "at meetings .related to health 

y concerns, building media skills. (This is the basis for continuing 
education. ) 

*53. "Others" might include: other health professionals, attendants or aides, 
( students in other professions, medical personnel, outside agency per- 
r sonnel, families. 0 
54. Ability to organize the task to be dene: give directions and instructions 
at levels suitable to OTR's, COTA's, aides, volunteers and others. Al- 
though this experience might not be available to every student, it is 
an important consideration. 
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AMERICAN OCCUPATIONAL THERAPY ASSOCIATION , INC. 
FIELD WORK PERFORMANCE REPORT ; , 



Ms. / Mr 

(Last) 

Sociai Security Number 

College or University 

Student level: Junior 

Type of Field Work : 

Facility and address 



(First) 



(Middle) 



(Maiden) 



Senior . 



Basic Master 's 



Certificate 



Number of weeks assigned 



Dates of placement : From 

Order of placement : 1 2 3 4 
Absences : Number of days • 



19 



To 



19 



out of 12 3 4 
Reasons for ' 



Amount of time made up 



DATA GATHERING 

s 1. Gathers necessary pre-test information or materials 

before interviewing or testing patient /client . . 

2. Selects evaluation method and /or tools which are ap- 
propriate to the patient /client 

3. Administers test/ interview /evaluation in a climate 
appropriate to the patient's/client's disability, age and 

j personality 

4. Administers test/ interview /evaluation correctly accord- 
ing to standardized or recommended technique 

5. Demonstrates competence in evaluation techniques/ tests 
by adapting metiiod to elecit data when standard or 
recommended technique is not possible 

6. Obtains additional or supplementary information from 
appropriate persons and available records 

7. Assesses patient's /client's physical, social and 
emotional needs 

TREATMENT PLANNING 

8. Defines problem areas for treatment using information from 
interviews and observations of patient /client as well as 
data from appropriate standardized or clinical tests .... 

9. Determines long-term treatment goals in accordance with 
/ probable discharge situation 

® American Occupational Therapy Assoc. Oct. 1973 

o 

ERLC 





Rare* 


Occa- 
sionally 


Fre- 
quently 


Consist- 
ently 


Comments 


















































• 






















(FOR CLARIFICATION OF ITEWS, SEE RATER'S GUIDE) 
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TREATMENT PLANNING ( CONT ' D ) 

10. Develops treatment plan in collaboration with patient/ 
client 


Rare- 
ly 


Occa- 
sional I 


Fre- 
y quenti) 


Consist 
f ently 


Comments 












11. Guides patient's/ client's selection of appropriate activ- . 
ities which will lead to goal achievement . 












12. Demonstrates ability to establish treatment priorities . . 

13: Presents several alternatives or innovative solutions to 
problems 






















TREATMENT IMPLEMENTATION 

14. Coordinates schedule with patient /client, other staff 


(FOR CLARIFICATION OF 


/ 

"ITEMS, SEE RATER'S GUIDE) 












15. Adheres to precautions 












16. Arranges equipment and materials according to treatment 
purpose 












17. Positions patient /client comfortably and appropriately . . 

18. Explains to patient / client what he will be doing and why, 
modifying plan, if possible, according to patient's/ 
client's reaction 























19. Intervenes at signs of fatigue or frustration, if appropriate . 

20. Uses praise or other reinforcers to encourage appropriate 
behavior . 






















21. Sets appropriate limits in response to undesirable phy-' 
sical or social behavior 












22. Selects or modifies available treatment environment to 
support patient's/ client's best performance 










9 


23. Establishes and maintains therapeutic relationship with 












24, Takes initiative to reevaluate the patient's/client's 
status at aporopriate intervals 












25. Demonstrates awareness tff the patient's/ client's 

status by making program changes in response to changes 
in the patient /client or his environment 












26. Informs and /or prepares the patient /client for program 












27 .Plans for patient's/client's discharge in adequate time . 












28. Demonstrates ability to terminate 4 -eatment appropriately . 












29. Evaluates the effectiveness of treatment procedure used . 










/ 


JO. Is alert to .needs of the total patient /client group .... 










t. 



_ ;J „„ ^MMVMW y, V/U H . . . . 
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TREATMENT IMPLEMENTATION ( CONT-' D ) 

31. Collaborates with others in promoting an atmosphere 
which will support the health and independent function- 
ing of the patient / client 


Rare- 
ly 


Occa- 
sionally 


Fre- 
quently 


Consist- 
ently 


Comments 












32. Considers the patient's/client's post -discharge program 
in relation to the treatment center and /or community 
resources ... 


1 

i 










33. Deals effectively with a variety of disability and age 
groups , . 












COMMUNICATION SKILLS 

34. Uses professional judgement in selecting interpreting 
and reporting data '. . 


(FOR CLARIFICATION OF ITEMS, SEE RATER'S GUIDE) 












35. Substantiate^; statements with data from evaluations, 
observations and interviews 










/ 


36/ Communicates effectively with staff and other people . . 
37. Communicates effectively with patients/clients 






















« * '• 

38. Presents oral reports in appropriate form, content and 
manner'. : 








i 




39. Produces written reports according to requirements of 
the facility . . 












40. Recognizes and utilizes non-verbal communication .... 
PROFESSIONAL CHARACTERISTICS 

41. Prepares daily work program for self, scheduling appro- 
priate amount of time for preparation and completion of 
tasks : . . . 












(FOR CLARIFICATION OF ITEMS, SEE RATER'S GUIDE) 












42. Maintains working area in a manner conducive to efficiency 
and safety 












43. Shares responsibilU-yTn maintaining proper level of 

supply inventory 












44. Assumes authority appropriately 












45. Adjusts pace to program requirements 












46. Asks appropriate questions when in doubt 












47. Modifies behavior accordingly in response to supervision . 

48. Works appropriately with persons of varied racial-, ethnic 
and sociological backgrounds 






















49. Handles personal and professional frustrations appro- 
priately . ; 












50. Recognizes own strengths and weaknesses 














PROFESSIONAL CHARACTERISTICS (CONT'D) 

51. Demonstrates active interest in areas other that occupa- 
tional therapy that are relevant to total health care 

52. Improves current level of skills and knowledge by 
independently participating in learning.experiences . . . 

53. Interprets occupational therapy to others according to 
thoir level of interest and understanding 



54. Demonstrates ability to supervise staff and/ or volunteers. 



ly 


Occa- 
sionally 


fre- 
quently 


Consist- 
ently 


Comments 








































/ 



(FOR CLARIFICATION OF ITEMS, SEE RATER'S GUIDE) 

ADDITIONAL COMMENTS 

I 

Comments might include nature of caseload, notable strengths and weaknesses and potential for work in 
tn i s area . 



I have read this report 



(Signature of student) 



: RIC 



(Date) 



Signature of rater 
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Position 

Number of persons contributing to report 



v* i 
" Qualitative Assessment of Field Work Competency j 

GUIDELINES FOR RATERS APPENDIX F - QAFWC 

Purpose 

This Qualitative Assessment instrument is being used only for research 
purposes as part of the current AOTA project which is designed to eval- 
uate measures of prof iciency' in Occupational Therapy, It is intended 
as a general measure of the qualitative aspects of* a student's fieLd * 
work performance and is to be used in conjunction with the existing 
AOTA Field Work Performance R-eport only during the course of the eval- 
uation project. 



Use of the-Rating Scale 

Although the instrument may appear somewhat lengthy, note that it includes 
only eleven major elements for rating. On Parts I, II and III the student's 
performance is rated on a scale of 1 - 5 ranging from 1 - VERY WEAK to ,5 - 
VERY STRONG. (See rating booklet for more detailed explanation,) Tor each * 
^vait to be assessed, be sure to read the entire page and review each of the 
behavioral indicators before vou make your rating, " 

Timing of the Rati ng 

The rating should be completed as close as possible to the end of the stu- 
dent's field work experience in your setting.^ UNDER NO CIRCUMSTANCES should 
the content of the form be shared with the -student prior to the completion 
of the rating. However, after* the rating is completed, the results may be 
shared with the student if you and/or the student so desires. 

Confidentiality 

As this instrument is for research purposes only, the ratings/ have no effect 
upon the student's field work performance grade and should ndt be transmitted 
to the student's school. As stated previously K resul ts may be sharedwith 
the student after the ratings are completed but, otherwise, confidentiality 
should be maintained. ,/ 



Reporting the Rating Results 

Upon completion, rating forms should be returned as soon as possible to the 
Professional Examination Service, 475 Riverside Drive, N. Y • , N.Y. 10027. 
A mailing label is enclosed for your convenience. 

Evaluation of the Rating Form 

The Qualitative Assessment instrument is being used in conjunction with the 
Field Work Performance Report on an experimental basis. Therefore, your , 
personal reactions regarding the u^e of this instrument will provide very 
important feedback to the project. Please complete the enclosed evaluation 
form and return it with the student racing forms. 
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RATER'S EVALUATION 0? THK QUALITATIVE AJ -SSmLsT OF FIELD WORK COMPETENCY FORK 



!► Plettno Indicate 
the indicators < 



■•nr wcale provided the use cent to which the terminology wie-d. 
each part of the tool) was c lear /in U« seanlng. 



Very 
CleJir 



in 



Uncicat 



IA. AppHc. of Evaluation Principles 

IB. Administration of Eva;. Procedure* 

IC. -Reporting and Recording 

ID. Interpretation and ?Jci limr ton of Datd 
HA. Coal Fomuljfitlon 
IXB. Developing .-and Presenting Treatment Plan* i 

IIIA« Organising the Treatment Progroo 1 
TUB, Carrying Out the Treataenr Pragmas'" 
II1C. /Therapeutic Reiaiir^hipB 

HID. Treatment Modification or fenslnaclon ' 

HIE, Communication Skills 
COMMENTS : 



3 
3 



3 
3 
3 
3 
3 
3 



I 4 



5 

5 
5 
S 

5 

W 5 " 



If (in response to the above question) you encountered terminology which was unclear 
or ambiguous,/ please indicate the terminology and its location on the form. (Li»t 
be .low. ) 



Terminology 



Part (Location) 



C<38SS£*nt« 
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(Note: Uae the back of this page to extend your il«t, If necessary.) 
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K The general CAirgorfe* {or traits) that the Jora attempts • -seas were derived 
in relation to those on the Field W?rk Per fortswmce Report* ;i; your opinion, are 
these the moat Important ftnj relevant traits to use for a qualitative assessment 
of OccupitHortftt Therapy iietd work performance? 



- *A. 


Alii") 1 i' i' f s f v a t as <n f 4 rf^ri 1^ f i : -** i *v 1 *v 


YES , 




' UNDECIDED 


I ? 


r roce<#wr *r !' 


YES 




^UNDECIDED 




ft < v pc* ? t jTif. ijno cor 1 


-. YES 





^UNDECIDED 




interpret^! tc»n *kf5d Utilisation of D.*v> 


YES 


_N fc 0 


UNDECIDED 


Mi 

uA,. 


Co (* ! Ft* r^ e . ; 5 :* • i 03$ 


YSS 


. NO 


; UMDECXDED 




Dev, ?*nd Presenting Tcc^t^ent Plans 


j r Yg£ ' 


^NO 


U>^DEC!OED 


? ? r * 


v * A », £ e > r e a "~-t* n * rr <u* r 


^ YE?' 


_N<) 


^DECIDED 


1.5 !». 


Cirryins; Out ''he 7rf»«WT.ent Pro£rAr\ 


VK:: 


Ji^ 


tfKDECIDED 


in*:. 


f**er«*p«ui J c r.e : . •.. iortahijifi 


Y>V 




tSJOEClDED 


i ! in. 


Treatment HtuJif* or Terrsinati^» 







' UNDECIDED 


ll'Sf.. 








l?N*£iECn>ED 


{V. 


perr.<M-.;^ & • v .»c tt-2 ; 31. 






_^'DECIDg!» 


If "NCV' 


'>■•/ « : 5>?^-Ves U- if- * ^*pi*:n :lr r-:V 




* i of. : i>: 





op*:i!> '*?'., -f:*i- v.^ - Lr:-j'; (.ulvr :: : 1 * * -:^e hppropr Har e 



. ' P , • , y - 

j - 3 - 

/ ' 
To what extent did you find the behavior Indicators a helpful guide in 
determining your ratings? 

Very Helpful Very Unhelpful 

1 2 3 4 - 5 

COMMENTS: 



1 



In the process of ranking the ratings, did you have any students whose behaviors \ 

did not fit the categories as presented^? *" .. . \ 

YES „N0 ' U NDECIDED 

If "YES" to the above, please explain: ^ 



We-c the instructions for using thtT'assessment fora clear to you? 

Very Clear Ve'ry Unclear 

• 1 2 3 4 _ 5 

COMMENTS : ' 



Hou uscfui was the assessment tool specifically in regard to YOUR fiald work setting? 

Very Useful * Very Ueglega 

1 2 3 U ' 5 
COMMENTS: " 



.J ' , 

- 4 - 

-if - . ~l 

9. On the average , approximately how long did it take you to complete the assessment 
form on a student? 

less than%15 min. 46 - 60 min.. more than 2 hrs. 

15 - 30 min. _1 - l 1/2 hrl. (Please specify time.) 

31 - 45 min. _1 1/2 - 2 hrs. 



10. This instrument wa| developed for use in the current project because occupational thera- 
pists have indicated a need for a qualitative assessment of^student field work perfor- 
mance. To what extent do you think the instrument meets that expressed need? 

Ve ^ Wel1 Not at All 

j 2 3 4 r 

COMMENTS: ; 



11. • Which of the following words characterize y6ur experience in using the Qualitative 
Assessment instrument? (Check as many as may be applicable.) 



_Very Useful 
_Easy to Use 
^Efficient 

Takes Too Much Time 

Ambigous 



_Needs Rev! ?ion 
"Difficult. • 
.Worthwhile 
_Accep table As Is 



Confusing , 

PERSONAL DATA (RATER) 

12. When did you firpt enter the practice or Occupational^Therapy? 

13. How long have you been a supervisor of student field work? 

14. Have you participated in continuing education programs in 
any of the following r.ontent areas? If so, indicate year. 



Good Qualitative Tool 

Imprac tical 

Hm ?e levant Content 
I'ieless 
Helpful 



Content 



\ 



Year of Participation 



Superyis i on 
Teste & Measurements 
Clinical Education 
Other (Please specify.) 



(Month, Year) 
(Year[s)) 



L5. 
16. 
17. 



How many O.T. students have you supervised within the past year? 

How many student field work placements can your facility provide each year? 

Approximately how many requests for student placements does y,our facility 
receive each year? 



[Please use the back of this page fci any additional comments, suggestions or reactions.) 
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Field Work Performance Assessment 
Occupational Therapy Assistant APPENDIX G - QAFWP 



GUIDELINES FOR RATERS 



Purpose 

This instrument is being used only for research purposes as part of the 
current AOTA project which is designed to evaluate measures of proficiency 
i^'/tfccupatiopal Therapy. It is intended as a general measure of the 
qualitative aspects of a student's field work performance and, in its 
present form, the instrument is to be used' only during the course of the 
evaluation project. j 

Use of the Rating Scale f 

Note that the instrument includes eight major elements for rating. For 
each trait or characteristic, the student's performance is rated on a 
scale of 1 -5 ranging from 1 - VERY WEAK to 5 - VERY^STRONG. (See rating 
booklet for more detailed explanation.) For ^ach trait to beias&essgd, 
be sure to read the entire page and review each of the behavioral 
indicators before you make your rating . The /student should demonstrate 
all of the behaviors indicated under c a given section of the rating scale 
(i.e., VERY WEAK, WEAK, MODERATE, STRONG, VERY STRONG) in order to 
warrant the corresponding numerical rating (i.e., 1, 2, 3, 4, 5). 

Timing of the Ratin g 

The rating should be completed as close as possible to the end of the stu- 
dent's field work experience in your setting, UNDER NO CIRCUMSTANCES 
should the content of the form be shared with the student prior to the 
completion of the rating. However, after the rating is completed, the 
results may be shared with the student if you and/or the student so 
desires. 

Confidentiality 

As this instrument is for research purposes only, the ratings have no effect 
upon the student's field work performance grade and should not be transmitted 
to the student's school. As stated previously, results may be shared with 
the student after the ratings are completed but, 'otherwise, confidentiality 
should be maintained. 

Reporting the Rating Results 

Upon comprieCXton, rating forms should^be returned as soon as possible to the 
Professional Examination Service, 475 Riverside Drive, New York, New York 
10027. 

*)> 

Evaluatioffof the Rating Form 

This Instrument ir, being used on an experimental basis. Therefore, your 
personal reactions regarding the use of the instrument will provide very 
important U:edbaek. Please^complete the enclosed evaluation form and 
return it with the student rating forms to the Professional Examination 
Service. 

i 
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RATER 1 S EVALUATION OF THE FIELD WORK PERFORMANCE ASSESSMENT 
(Occupational Therapy Assistant) 



. .Please indicate on the .scale provided the extent to which the terminology used in 
the indicators (for each part of the tool) was clear in its meaning. 

Very Very 
Clear Un clear 

IA. Collecting Evaluation Data 1 2 3 4 5 

IB. Communicating Evaluation Findings 1 2 3 4 5 

II. Assisting in Treatment Planning 1 2 3.4 5 

IIIA. Organizing and Carrying Out 1 2 1 '3 4 5 

Treatment Program 

r 

HiB. Establishing Patient/Client 1 2 3 4 5 

Relationships 

IIIC. Assisting with Program Modification 12 3 4 5 

HID. Communicating Effectively 1 2 3 4 5 • 

IV.. Role Expectations 1 2 3 4 5 

COMMENTS ; 



2. If (in response to the above question) you encountered' terminology which was unclear 
or ambiguous, please indicate the terminology and its location on the form. (List 
below.) 

Terminology " Part (Location) ~| Comments 



(Note: Use the back of this page to extend your list, if necessary.) 



2 - 



3, The general categories (or traits) that the form attempts to assess were derived 
from those most frequently included on field work performance " forms currently 
used by individual OTA educational programs. In yqur opinion, are these the 
most important and relevant traits to use for a qualitative assessment of 
Occupational Therapy Assistant field work performance? 



IA. Collecting Evaluation Data 

IB. Communicating Evaluation Findings 
II. Assisting in Treatment Planning 

IIIA. Organizing and Carrying Out 
Treatment Program 

IIIB. Establishing Patient/Client Relationships 

IIIC. Assisting with Program Modification 
HID. Communicating Effectively 

IV. Role Expectations 



YES 



_YES 
YES 



_YES 
_YES 
_YES 
YES 



_N0 
_N0 
_N0 
NO 



_N0 
_N0 
_N0 
NO 



UNDECIDED 



UNDECIDED 



JJNDECIDED 
UNDECIDED 



JUNDECIDED 
UNDECIDED 

JJNDECIDED 
UNDECIDED 



If "NO" to any of the above, please explain and/or make suggestions for additions 
or deletions: 



The behavior indicators presented to guide'you in your rating of each trait or 
category were selected as a sampling of the most important indicators . In your 
opinion, were the indicators selected for use appropriate? 



Very 

Appropriate 



Very 

Inappropriate 



COMMENTS: 



ERLC 
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5. To what extend did you find the behavior indicators a helpful guide in 
determining your ratings? 

Very Helpful Very Unhelpful 

1 .2 3 4 5 

COMMENTS : j 

— j ; 



— : — ^ ; ■ — 

In the process of making the ratings, did you have any students whose behaviors 
did not fit the categories as presented? 

YES NO UNDECIDED 

If. "YES" to the above, please explain: 



7. Were the instructions for using the assessment form clear to you? 

Very Clear * Very Unclear 

1 2 3 4 5 

r 

COMMENTS : 



8. How useful was the assessment tool specifically in regard to YOUR field work setring? 
Very Useful Very Useless 

1 2 3 4 5 

COMMENTS: 



ERIC 
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On the average, approximately how long did it take you to complete the assessment 
form on a student? 



less than 15 mir.. 46 - 60 min. more than 2 hrs. 

_ 15 - 30 mm. ; ; i . ^ h rs. (Please specify time.) 

31 - 45 min. 1^ . 2 hrs. ; 

This experimental instrument constitutes an attempt to develop a tool which could 
assess the field work performance of any Occupational Therapy Assistant student. 

To what extent do you think the instrument achieves that objective? 

Ver y Wel1 Not at All 
12 3 4 5 

COMMENTS : i 



Which of the following words characterize your experience in using the OTA Field 
Work Performance Assessment instrument? (Check as many as may be applicable.) 

Very Useful Needs Revision Good Qualitative Tool 

Eas y t0 Use Difficult Impractical 

Efficient Worthwhile Relevant Content 

T akes Too Much Time A cceptable As Is _Useless 



^Ambiguous C onfusing H elpful 

PERSONAL DATA (RATER) 

When did you first enter the practice of Occupational Therapy? (Month, Year) 

How long have you been a supervisor of student field work? ^(Years) 

Have you participated in continuing education programs in any 
o<: the following content areas? If so, indicate year. 

Content year of Participation 
Supervision 

Tests & Measurements - 

Clinical Education — — — 

, Other (Please specify.) 



How many 0. T. Assistant students have you supervised within the past year? 

✓ 

How many student OTA field work placements can your facility provide per year? 

Approximately how many requests for student placements does your facility 
receive each year? 



Please use the back of this page for,, any additional comments, suggestions or 
reactions youyAay have. 

.. . • c 
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The Qualitative Assessment of Field Work 
Competency rating booklet l ».as been removed 
by the forwarding agency - AOTA. The reade 
may wish to contact that organisation for 
further information regarding the tool. 
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APPENDIX H 



DATA SUMMARY 
RATERS* EVALUATIONS 
of the 



\ 

\ 



QUALITATIVE ASSESSMENT ,0F FIELD WORK PERFORMANCE INSTRUMENT 



for 

OCCUPATIONAL THERAPISTS 



t 

/ 



Evaluation of i->. gures of Proficiency in Occupational Therapy 

July, 1976 



DATA SUMMARY: RATERS' EVALUATIONS 5 OF QAM (OT) 

i 

Qt 11 TZ 2,? 0I ? e SCa i e Pr ° Vided the fiXtent t0 ^ich the terminology used 
in the indicators (for each part of the tool) was clear in its meaning. . 



Rating Scale 







Very Clear 












Very Unclear 












i 




2 




3 




4 




5 . 










if 


L 


1 


7, 


'•# 




1 


I 


■ 

' 1 


V 


N= 


Mean 
Rating 


1A, 


Applic. of Eval. Prin. 


JLOU 


kl 9 


50 


21.0 


18 


7,6 


9 


3.8 


1 


.4 


238 


1.49 


IB. 


Admin, of Eval. Proc. 


10J 


00, j 


47 


19.7 


20 


8.4 


7 


2.9 


1 


.4 


238 


1.47 ' 


IC. 


Reporting & Recording "' 


175 

X 1 J 


71 S 


33 


13.9 


19 


8.0 


8 


3.4 


3 


1.3 


238 


1.45 


ID. 


Inter. & Util. of Data 




79 n 


40 


16.9 


15 


6.4 


9 


3.8 


) 2 


.8 


236 


1.45 


IIA. 


Goal Formulation 


165 


69.6 


41 


17.3 


19 


8.0 


10 


4.2 


2 


g 


237. 


1. 50 




Dev.. & Pres. Tr. Plans 


176 


73.9 


39 


16.4 


' 11 


4.6 


' 8 


3.4 


4 


1.7 


238 


1.42 


IIIA 


urganiz. ir* Prog, 


175 


73.8 


37 


15.6 


15 


6.3 


6 


2.5 


4 


1.7 


. 237 


1.43 


IIIB. 


Carrying Out Tr, Prog. 


175 


73.5 


35 


14.7/ 


' 15 


6.3 


10 


4.2 


3 


1.3 . 


238 


1.45 


IIIC. 


Therapeutic Rel. 


176 


73.9 . 


'31 


13.0 


16 


6.7 


7 


2.9 


' 8 


3.4 


238 


1.49 


HID. 


Tr ( Mod, or Term, 


149 


63.1 


41 


17.4 


31 13.1 


11 


4.7 


4 


1.7 


236 


1.'64 


HIE. 


Communic. Skills 


175 


74.5 


31 


13.2 . 


14 


6.0 


6 


2.6 


9 


3.8 


235 


1.46 


Total 




1,859 


71.5 


425 


16.4 


193 


7.0 


91 


3.5 


t 

41 


1.6 







Overall Mean Rating: 1.32 
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.DATA SUMMARY: RATERS' EVALUATIONS OF QAFWP.(OT) 



Q. 3. The general categories (or traits) that the form attempts to assess were 
derived in relation to those on the Field Work Performance Report. In 
your opinion, are these the most important and relevant traits "for a 
qualitative assessment of Occupational Therapy field work performance? 



YES NO UN- NO 

DECIDED, RESPONSE 



1 




' i 


I 


n 

V 


% 


V 


ft 

I 


11 

!/ 


/. 


IA. 


.Applic. of Eval.'Prin, 


223 


rt ft ft 

88.8 


3 


1.2 


ft 

8 


3,2 


it 


t 

L ft 
0,0 


IB. 


Admin, of Eval. Proc. 


218 


86.9 


6 


2.4 


9 


3.6 


18 


hi 


IC. 


Reporting & Recording 


223 


88.8 ' 


■6 


2.4 


■»i 


1.0 


18 


7„2 


ID. 


Inter, k U til. of Data 


224 


89,2 


2 


.3 


8 


3.2 


17 


6.8' 


IIA. 


Goal Formulation 


227 


90.4 


2 


.8 


• 7 


. U „ 


15 


6.0 


IXB. " 


Dev. & Pres. Tr. Plans 


229 


91.2 


2 


.8 


4 


1.6 

y 

2.4 


• 16 


6.4 


IIIA. 


Organic Tr. Prog. 


227 


90.4 


2. 


.8 


6 


16 


6.4 


IIIB. 


Carrying Out. Tr. Prog. 


225 


39.6 


4 


1.6 


6 


2.4 


16 


6.4 


me. 


Therapeutic Rel. 


220 


8?; 6 


9 


3.6 


6 


2.4 


16 


6.4 


hid. 


Tr. Mod. or Term. 


213 


84.9 


10 


4.0 


13 


5.2 


15 


6.0 


HIE. 


Communic. Skills 


230 


91.6 


4 


1.6 


3 


1.2 


U 




IV. 


i 

Personality Characteristics 


202 


80.5 


u 


4.4 


24 


9.6 


34 


5.6 



2,661 90.1 61 2.5 98 5.4 192 2.0 

I 



trxi; or c»t.. f ..ry v*f- <i*cu^ = ■■■ -.a^a irv • : . i ■ 

selected! fo? Mft« 4*{5|? • •:*.",.•:• 



V*:, 



In th« jjfoci-sa of raking snt ^H^5»"'ii<! *«»« nnv >,*«<:.•<•.'. 
whose behavior ft noV fit ?h« «»i^:ors«»' $U *»e«!M? 



0 




How v- Sf.;i t; I u?L:. /, h«: ,i ?ot-i :> J" 0-'. ; i : c a : * *.' i. v. a r Y0 5 -T 



- »« . ' 



0?; • r.i;;v.» ' aj?;:^ :'";ir^'cl'v h;;w Ion,; dVj i u^v yo;: no complete 

U::^ than 15 - "i; . M ,,s ^ r \ „ \» i*. . ; -ore than 

3 :; snLn. . rt.£n. r'U.tJ. V.r;:- hr;-. 2 hrs. 



SO, Thif. i/n:- irvr^ru cov* I of>*tf fc: cst'.irs Uu. current pro^;-r- 

bfcCwiu :»<:-. occupy t *am*I tht>: \!ip i $z *; Uv v * n> eVi for 

^u4& i i ai jv<' aasftsssacnt of ^•udt-ni :V5 work perl enhance-. To 
wtaat exi£n£~-&i you thinK, th« instrument fciKtf.ttf fclha" expreissec' 

• . ! - • 

, Very WcH ' " ' - ' Not. At. Ai 1 

L " * \ 

\ - 
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Which of the following words characterize your experience in 
using the qualitative assessment instrument? (Check as many 
as may be applicable.) 



N « 106 



NO RESPONSE 



\ 



Positive 



Relevant Content 


IS! 


Easy to Us* 




Good Ouai % Tool 


1 


Helpf * 




Wort.- . i 1 e 


' }'K> 


Efficient 




?e/y .ireful 


9 1 


Accept. rifti/ A « 


•«.3 


Total 


r 1 ' 1 



Negative 

Needs evi sion 52 

Ambiguous 24 

Takes " r--. Much Time 23 

Confu* 19 

Imp i -ft-r. t ; ; s i 14 

Useless 10 

Difficult 7 



Total 149 (147.) 



> : or -.!:; < Vr.ec ked: 1 , f;4 



• Cv^s Revision' 



exc 1 uded , r.e;-.^ t i ve is reduced 



V 
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DATA SUMMARY 
RATERS' EVALUATIONS 
of the 

QUALITATIVE ASSESSMENT OF FIELD WORK PERFORMANCE INSTRUMENT 

for 

OCCUPATIONAL THERAPY ASSISTANTS 



r 



Evaluation of Measures of Proficiency in^p^cupational Therapy 

J ' July, 1976 
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DATA SUMMARY: RATERS' EVALUATIONS OF QAFWP (OTA) 



Q. 1. Please indicate on the scale provided the extent to which the terminology used in the 
indicators (for each part of the tool) was clear in its meaning. 



Very Clear 



Rating Scale 



1 



3 



.Very Unclear 







// 


X 


i 


I 


1 


I 


// 




i X 




Mdfln 

nedn 
Ratii 


IA. 


Collecting Evaluation Data 


117 


70.1 


36 


21.6 


5 


3.0 


5 


3.0 


4 2.4 


1/ u 


1 4A 
1.40 


IB. 


Communicating Evaluation Findings 


119 


71.7 


33 


19.9 


5 


3.0 


5 


3.0 


4 2 4 


l/U 




II. 


Assisting in Treatment Planning 


116 


69.5 


33 


19.8 


9 


5.4 


6 


3.6 


1 1 fi 


1 7(1 

I/U 


1 /.A 

1.49 


IIIA. 


Organizing and Carrying out 
Treatment Program 


119 


71.7 


27 


16.3 


9 


5.4 


6 


3.6 


5 3.0 


170 


1.50 


IIIB. 


Establishing Patient/Client 
Relationships 


118 


71.1 


32 


19.3 


7 


4.2 


3 


1.8 


6 3.6 


170 


1.48 


inc. 


Assisting with Program 
'Modification * 


119 


71.7 


34 


20.5 


4 


2.4 


4 


2.4 


5 3.0 


170 


1-45 


HID. 


Communicating Effectively ' 


116 


69.5 


34 


20.4 


8 


4.8' 


5 


3.0 


4 2.4 


170 


1.49 


IV. . 


i 

Role Expectations 


102 


62.2 


34 


20.7 


16 


9.8 


6 


3.7 


6 3.7 


170 


1.66 
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DATA SUMMARY: RATERS' EVALUATIONS OF QAFWP (OTA) 



Q. 3. The general categories (or traits) that the form attempts to assess were derived from 
those most frequently included on field work performance forms currently used by indi- 
vidual OTA educational programs. In your opinion, are these the most important and 
relevant traits to use for a qualitative assessment of Occupational Therapy Assistant 
field work performance? 



UNDECIDED NO YES NO 

RESPONSE 







1 


I 


i 


X 


I 


Z 


f 


I 




IA. 


Collecting Evaluation Data 


13 


7.6 


9 


5.3 


144 


84.7 


4 


2.4 


170 


IB. 


Communicating Evaluation 


8 


4.7 


10 


5.9 


148 


87.1 


4 


2.4 


170 


II. 


Assisting in Treatment Planning 


5\ 


2.9 


3 


1.8 


159 


93.5 


3 


1.8 


17.0 


IIIA. 


Organizing and Carrying Out 
Treatment Program 


2 


\.2 


2 


1.2 


163 


95.9 


3 


1.8 


170 


IIIB. 


Establishing Patient/Client 
Relationships 


1 


0.6 






166 


97.6 


3 


1.8/ 

i 

i 


170 


inc. 


Assisting with Program 
Modification 


8 


4.7 


4 


2.4 


152 


89.4 


6 


3o 

/ 

i 


170 


hid. 


Communicating Effectively 


1 


0.6 


3 


1.8 


163 


95.9 


3 / 

/ 


1.8 


170 


iv.- 


Role Expectations 


12 


7.1 


5 


2.9 


148 


87.1 


/ 


2.9 


170 



■ / 

/ 



Q. 4. 



The behavior indicators presented to guide you in your rating of each trait 
or category were selected as a sampling of the most important indicato rs. 
In your opinion, were the indicators, selected foT~use appror.riati* 



Very 
'Appropriate 



83 (48.8%) 
No Response: 



51 (30.0%) 
9 (5.3%) 



20 (11.8%) 



Very 
1,^/ppropriate 

4 5 
5 (2.9%) 2 (1.2%) 

Mean Rating: 1.71 



N = 170 



Q ' 5 " SnSl'yS «t£J.r flnd behaVi ° r indicatora a he W Buide in deter- 



Ver? Helpful 
1 



81^(47.6%) 50 (29.4%) 22 (12.9%) 

No Response: 6 (3.5%) 



Very Unhelpful 
4 5 
10 (5.9%) ] (0.6%) 

Mean Rating: 1.78 



N « 170 



Q " 6 ' JnrfH,3 rOCeS ^.-° f r klng thG ratin 8 s » did y°" have any students whose behav- 
iors did no_t fit the categories as presented? 



,YES 
12 (7.1%) 
No Response: 9 (5.3%) 



NO 

96 (56.5%) 



UNDECIDED 
53 (31.2%) 
Mean Rating: 2.26 



N = 170 



Q. 7 Were the instructions for using the assessment form clear to you? 

\ 

V ery Clear 

1 2 1 



106 (62.4%) 41 (24.1%) 14 (8.2%) 

/ 

No Response: 6 (3.5%) 



Very Unclear 
4 . 5 

2 (1.2%) 1 (0.6%) 

Mean Rating: 1.48 



N = 170 
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Fo*. v v?:ui was the assessment tool specifically in regard to YOUR field work 

set 5. i-iig? . % * . 

V ery Useful ^ Vc:x y' Useless 

V 

1 . * 3 4 "v. 5 170 

68 (40.0%) 55 (32.4%) 27 (15.9%)' 11 (6.5%> ! (0.6%) 

No Response: 8 (4.7%) Mean Rating: 1.50 



Q. 9. On the average, approximately how long did it take you to complete the assess- 
ment form on a student? 

less than 15 - 30 31 - 45 46 - 60 I - 1 1/2 11/2-2 more than 

15 min. min. min. min. hrr,„ hrs. 2 hrs. 

13,(7.6%) 76 (44.7%) 47 (27 .o%) 13 (7.6%) 9 (5.:;%) 6 (3.5%) 1 (0.6%) 

No Response: 5 (2.9%) € Mean Rating: 2.70 „ N = 170 

Q.10. This experimental instrument constitutes an attempt to develop a tool which could 
assess the field work performance of any Occupational Therapy Assistant student. 
To what extent do you think the instrument achieves that objective? 

i 

Very Well Not At All 

1 2 . * ' - 3 4 . 5 

56 (32.9^) 73 (42.9%) - 24 (14.1%) 7 (4.1%) 1 (0.6%) N = 170 

No Response: 9 (5.3^) ^ Mean Rating: 1.91 
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Q, 11. Which of the following words characterize your experience in' using the OTA 
Field Work Performance Assessment instrument? (Check as many as may be 
applicable.) J 



N - 170 



Positive 



Very Useful 62 

Easy to Use 104 

Efficient 87 

Worthwhile 89 

Acceptable as Is 36 

ISood Qualitative Tool 231 

Relevant Content 115 

Helpful 90 



Total 



864 (89Z) 



Negative 



"fakes Too Much Time 


6 


Amb iguous 


18 


Needs Revision 


60 


Difficult 


6 


Confusing 


11 


.Impractical 


3 


Useless 


8 


Total 


112 



112 (11%) 



\ 



6 



I 



V 
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The Field Work Performance. Assessment - Occupational 
Therapy Assistant rating booklet has been removed by 
the forwarding agency - AOTA. The reader may wish to 
contact that organization for further information re- 
garding the tool . 
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